Welcome to Photo.net: A Community of Photographers

Community > Forums > Digital Darkroom > Software>Utilities > Ingesting 8 cards at once......

Ingesting 8 cards at once... identifying the readers?

Marc Rochkind , Jun 25, 2008; 09:35 p.m.

A while ago I said I was working on ingesting from multiple card readers, and now I've got it working. But, there's still the problem of identifying the readers. I have 8 Kingston 19-in-1 readers, and when they're connected to my Mac under OS X they do not appear at all. The volume is mounted only when a card is inserted. My app can get both the card label ("NO NAME", "NO NAME 1", etc.) and the reader description ("Kingston FCR-HS219/1" eight times), but they are of no use in telling one reader/card combination from another.

A message to the user of the form "You may now remove card NO NAME 1 from Kingston FCR-HS219/1" would obviously be very dangerous, and so my app says no such thing. It waits until all 8 (or whatever) are ingested.

The situation on Windows is somewhat different. A plugged-in reader does show up as a drive letter. Several, in fact. But the drive letters change as readers are removed and reconnected, the machine is rebooted, etc. Card labels are also accessible, but they are no better at identifying specific cards than they are on OS X.

If you format a card on the computer, you can set the label, but for several reasons I have been one of the many people who advise against formatting on the computer.

Does anyone know of a reader that is self-identifying, so that if you bought 8 of them they would have different descriptions? The only reader than accepts multiple cards of the same type I know of is the Delkin ImageRouter, but I'm told that it just shows up 4 times with the same description. Not even Slot A, Slot B, Slot C, and Slot D. Can anyone confirm?


Ingesting 8 cards simlutaneously

Responses

ian murren , Jun 25, 2008; 10:57 p.m.

I'm failing to see the point. But your best bet would probably be to name the cards, so when the mouth you see the names, and write the names on the portion of the SD cards that sticks out.

ian murren , Jun 25, 2008; 10:59 p.m.

Edit: � so when they mount�

Brad - , Jun 25, 2008; 11:05 p.m.

I don't get it.

You give the cards different names; the readers don't have names. At least on a Mac...

Peter Dove , Jun 26, 2008; 12:36 a.m.

The problem is that an application that is simultaneously reading from several different pieces of media will finish at different times. It would be nice to let the user know which one he could replace with a fresh card so the application could continue its batch "ingestion" job. The problem is two-part: the programmer has to identify the device handling the card in question, and then the program must give the user a way to identify the card that needs to be removed.

Marc, I suggested one or two things in response to your original post - did any of them help? From a user's standpoint I kind of like the idea of making any LED that might be on the reader blink when the app is done with it - that should be an easy and harmless but inelegant repetitive read hack. Am I right to presume you are just grabbing the "/Volumes/CARD_NAME" (Mac) or "Q:" (Windows) mountpoint name and parsing the directory tree for image files to ingest? You should be able to get unique device IDs from the device driver - after all, that's what the OS does to satisfy read and write requests to different devices. Each card is mounted from a sequentially numbered /dev/diskN (Mac) or \devices\abc:xyz:disk:N (or some such on Windows) device node, so you should be able to do something with that disk ID. Again, as with Windows, the device nodes will be numbered in the order that cards are inserted, so there is no persistence across card mount/dismount or machine reboots.

With regard to naming the cards, Marc points out that it is better to not format them on the computer but to let the camera do that. I don't know whether other cameras may be different, but Canon DSLRs give any card the same name: "EOS_DIGITAL". In Marc's scenario you would then end up with 8 cards with the same name attached to 8 readers with the same name.

Brad - , Jun 26, 2008; 12:49 a.m.

>>> In Marc's scenario you would then end up with 8 cards with the same name attached to 8 readers with the same name.

How bout this. You have 8 cards. Put card#1 in, select the card's name "EOS_DIGITAL", and type "1". Repeat for card#2, except type "2". Ditto through card#8.

Then just drag all the files from each card to a folder. That will cause 8 simultaneous copy operations; each ending at different times depending on card contents.

Robert Chura , Jun 26, 2008; 12:51 a.m.

I fail to understand why you want so many readers.

On my Windows PC I can rename each drive but only when a drive or card is inserted. When it is reinserted the name shows. I would think the same applies to Mac. An unmounted drive doesn't know what is to be there.

Marc Rochkind , Jun 26, 2008; 01:33 a.m.

I guess I didn't explain the problem very well.

Any scheme that involves putting cards in slowly, one-by-one, and waiting for them to mount won't work. It's too timing dependent, and that's not the way busy photographers with several cards to ingest work. The scheme has to be absolutely foolproof. Insert the cards, press a button, and go.

I did think over Peter's suggestions from last time, but I don't think the schemes are workable. In use, the lights sometimes blink and sometimes not. Going by a certain pattern isn't reliable enough... too much risk that the wrong card might be ejected. Anything based on order of insertion is too flakey. One slip up and the images could be ruined if the wrong card is removed.

There are indeed device IDs, but they identify the driver "minor number" assigned to the drive, and aren't fixed. No way to physically label the reader with a device number that would be constant. (Unless reader manufacturers assign a unique description, of course, which they currently do not.)

Brad's idea is to manually ingest the cards. Again, not suitable for a high-speed production environment.

Why have 8 readers? So that a photographer coming back from a shoot with 8 cards can just stick them all in, press a button, and go off to do something else while they ingest. Otherwise, he or she would have to sit there and monitor the ingestion until the 8 cards were dealt with.

Giving cards different names would actually work, since the names do show up. However, are there cameras that do that? None that I personally own do so. Does anyone have a camera that allows labeling of the card?

Formatting on the computer has other risks. While it solves the naming problem, it potentially causes other, worse problems.

Again, I need to emphasize: Any naming scheme has to be 100% reliable and foolproof even when the user is not concentrating on what's happening. Otherwise, if the wrong card is removed, it's disaster. Because ingestion is the first step in a digital workflow, there is no room for error. (Compare the consequences of an ingestion error to a printing error.)

The reality is that card makers, reader makers, and camera makers seem to have paid no attention to high-production workflow.

Marc Rochkind , Jun 26, 2008; 01:40 a.m.

I forgot to say why I'm even introducing this topic. It's an attempt to start raising this as a problem. If Lexar, Sandisk, Kensington, Delkin, and the others think their devices are OK, then perhaps this might be one small attempt, in combination with many others, to at least make that debatable.

(Some cameras have serial numbers that appear in the EXIF information, as maker notes. Most don't. But at least those that do provide a way to distinguish, say, one Leica M8 from another. Too bad the card reader makers don't do the same thing. That's all that would be needed.)

Brad - , Jun 26, 2008; 02:14 a.m.

>>> Brad's idea is to manually ingest the cards. Again, not suitable for a high-speed production environment.

Why not? A person has to manually insert the cards into a card reader. Is there some type of automatic ingestion possibility? Changing names to a number, maybe adds 2 seconds. Very small compared to the ingestion time.

>>> Any naming scheme has to be 100% reliable and foolproof even when the user is not concentrating on what's happening.

If you have 8 cards, each named "1" through "8", that seems straightforward. I just tried it with three cards - seems to work...

Is there a better naming scheme, that's more reliable, perhaps?

Brad - , Jun 26, 2008; 02:19 a.m.

>>> Changing names to a number, maybe adds 2 seconds. Very small compared to the ingestion time.

In fact, that card renaming operation can probably be automated with an AppleScript or Automator action. Each time a new card is inserted, the action fires off and changes the card's name.

In fact, I bet an action/script can be written that would automate the whole thing - including telling you to remove a particular card when it's contents have been copied.

Edward Ingold , Jun 26, 2008; 07:51 a.m.

A PC/Windows assigns drive letters to the READER, which identifies the card. I'm not sure if there is any significant advantage to reading two or more cards at once. There is just so much I/O bandwidth available for disk operations and reading two cards at once takes nearly twice as long. Besides, I don't think reading the cards is the bottleneck in image processing.

I am not familiar with MAC operation. If I were to believe the typical MAC user, they are not bound by the laws of physics :-)

Patrick Lavoie , Jun 26, 2008; 09:02 a.m.

"..that's not the way busy photographers with several cards to ingest work..""

Sorry but serious pro photographer have 3 thing that doestn include 8 reader to help;

1_a digital assistant

2_multiple high capacity CF card

3_a firewire 800 reader that could copy a full 8gig in approx 6min.

Having 8 reader wont change the fact that it took some time to copy a card, of course one will finish before the other one..you put it before?!

I dont see the point for a pro to go or use such a non snese system that are more prone to error as you demonstrate yourself. But if it can help you playing around with a solution, i think that by naming the card accordingly like card_01, card_02 etc..and making sure that you put the same number on the card reader itself, so people could associate by name or color code the exact combo.

I also think (not sure) that if you put multiple device on the same port, or on the same serie of port you will decrease the speed of this port, meaning that what should have took you 1 min to copy will take (for example) 2 min because or your system..i could be rong because i dont know that for sure, as im too busy taking picture VS doing test with 8 readers ; )

Marc Rochkind , Jun 26, 2008; 09:47 a.m.

Comments in response to Edward and Patrick:

Ingesting multiple cards at once does indeed pay off. In one test, I ingested 4 cards in 2 min. 2 sec., and 1 card in 1 min. 55 sec. (Transfer from cards, even with fast readers, is so slow that the computer is pretty much idle waiting for read requests to complete. Makes sense to have multiple requests being worked on instead of just one.) When there's a lot of processing, such as DNG conversion, the win is less, but even then it's substantial, especially if there are multiple cores/CPUs.

The way ImageIngester does this is absolutely not prone to error. One loads the card readers, starts the ingestion, and then removes the cards when it's finished. What would be prone to error, absent any way to reliably identify the readers, would be a system (not implemented) in which one removed one or more cards BEFORE they all finished. The potential for doing that is what I am researching.

Copying 8GB in 6 min. is fine, but if you have 4 such cards there's a big difference between:

1. Loading a card and coming back to it 6 min. later to load another, and

2. Loading all the cards, starting the ingestion, and then going off to do something else. When you come back (anywhere from 20 min. to a week later), you're ready to begin rating, selection, etc.

(I have users with a lot more cards to ingest... 20 - 30 is not unheard of.)

Anyway, my initial post wasn't to debate ingesting multiple cards, since this is entirely optional, and the one-by-one method is still there. It was to ask if anyone knew of any reader that was self-identifying, to ask about experience with the Delkin, and, in my second post, to ask if anyone knew of a camera that allowed one to label a card.

Marc Rochkind , Jun 26, 2008; 09:50 a.m.

Clarification: The 2 min. 2 sec. for 4 cards wasn't per card. That was the total time, the equivalent of 30 sec. per card.

Patrick Lavoie , Jun 26, 2008; 10:00 a.m.

Understand your point, and seem fine for a pro that use smaller card and doestn have lots of them so he need to empty them at once rather than one by one.

Like i said, normally a pro will have a digital assistant, but if not your software seem like a good alternative.

To answer your original questions;

On a Mac i dont think a self identifying card reader exist, since it doestn appear on the desktop. Only the card will mount. As for if a camera could name a card, again im not sure, but once on the desktop for the first time its easy to name them.

Edward Ingold , Jun 26, 2008; 10:07 a.m.

I believe I adequately addressed the identification problem - the O/S identifies the card reader, not the removeable medium. In multi-slot readers, each slot receives a separate drive letter.

Nikon cameras (and probably others) have the option of renaming the base file name for each image. This identifies the camera, but is not convenient to identify a particular job. That task I accomplish by creating a directory named per sortable date code (yyyymmdd) and topic description.

Marc Rochkind , Jun 26, 2008; 10:30 a.m.

Patrick--

There's no need to identify the reader prior to having the card mounted. Getting the label at that time is fine. What's not fine is that all the reader descriptions are the same.

I just thought of another problem with identifying the card and not the reader: When inserted, a CF card is out of sight. SD cards at least stick out a little.

I just plugged my 8 Kingston 19-in-1 readers into my Vista machine, and it could not mount them all. Each reader takes up 4 letters (even without any cards inserted). That's 32 right there. So, it seems that on a Windows machine one can't use a reader with so many slots. (Not a problem, as long as one knows in advance what to buy.)

I understand that can mount a reader on Windows instead of using a drive letter (haven't tried it), but that seems too cumbersome for a removable device. Maybe not... will have to see.

Ellis Vener , Jun 26, 2008; 12:15 p.m.

Marc,

They are not going to solve your problem but at least it will liven up your desk!

(link)

Chris Newell , Jun 26, 2008; 04:16 p.m.

Marc, I'm clueless to the process you are trying to do, but decided to google something and came up with the following link:

http://www.pictureline.com/products/12923/Lexar_Pro_CF_USB_2.0_Card_Reader/

to quote:

<<Lexar Pro CF USB 2.0 Card Reader Overview

Lexar Professional CompactFlash Readers (USB 2.0 and Firewire) are designed for pro photographers and serious hobbyists who are looking for maximum workflow speed with professional level durability and looks. The CompactFlash Reader will make downloading CF cards your from important photo sessions a breeze.

The CompactFlash Readers are stackable and designed to connect multiple readers to a single connection. Up to four Firewire CompactFlash Readers can be connected using Firewire daisy-chaining and up to four USB 2.0 CompactFlash Readers can be connected to the specially designed Lexar Professional 4 Port Hub USB 2.0 for fast, simultaneous CF card downloading.

Lexar Professional CompactFlash Readers support Lexar's exclusive ActiveMemoryル (AMS) and LockTightル technologies that enhance the features in your digital cameras and media processing software. With AMS, important settings can be saved to the card to prevent reentering or retyping these settings each time. With LockTight, you can secure LockTight cards to prevent data from falling into unwanted hands.

Features Incredibly fast data transfer Rugged black housing with textured rubber accents Stackable design LED indicator that lights up Available for Firewire or USB 2.0 Supports new Lexar technologies Active Memory and LockTight

Benefits Takes full advantage of the read/write capabilities of your CF card, speeding your workflow and saving you time and money Resists scratching and is more stable when inserting your CF card Easily stack and connect up to four readers for simultaneous downloads of large photo sessions Makes it easy to tell when your data transfer session is finished and when your reader is recognized by the host computer Ensures high-speed compatibility with your computer system whether you prefer USB 2.0 or Firewire devices Makes sure you'll be up and running quickly with Lexar's new advanced technologies>>

Perhaps the 'AMS' feature of the reader will do what you are looking for? It certainly appears that you would know when the different cards have finished being read.

There's some further info here: http://www.lexar.com/newsroom/press/press_02_27_06c.html that makes it sound like it might work for you?

And then again, this link makes me think it might not: http://www.robgalbraith.com/bins/content_page.asp?cid=7-6459-7252

And you may already know about all of this anyway.

HTH

Chris

Alex Meyer , Jun 26, 2008; 05:03 p.m.

Why not just buy 8 different readers, either different models or different manufacturers or both. Then they'll look and identify differently because they are different. If the physical differences are subtle, use some colored tape or something.

Marc Rochkind , Jun 26, 2008; 09:12 p.m.

Elllis-- If these guys are as smart as I think they are, then maybe they can just do the whole ingestion...

Marc Rochkind , Jun 26, 2008; 09:17 p.m.

Chris-- Thanks for this link. I don't think AMS is relevant, but maybe they have something else.

Alex--That was my original idea, which I mentioned in a post a week or so ago. It's still something I'm keeping in mind. But I think the Delkin ImageRouter may be the first of a series of designs that gang multiple readers together, and it appears that the slots on the Delkin are indistinct.

I also got into the depths of readers on Windows (my work up until now in this area was on OS X), and I see some numbers that very from reader-to-reader, which might be serial numbers. Not sure yet.

Anyway, the ideas here have been useful, and with further research I may be able to come up with something.

Marc Rochkind , Jun 26, 2008; 09:32 p.m.

Chris-- It seems that Active Memory may be defunct. I can't find any references to it later than 2006. It does seem that it would have allowed card-specific data to persist over camera reformats.

Nothing I see so far in Lexar's stackable readers suggests that there is a way to identify individual readers, but I'm still researching that.

Alan Peed , Jun 26, 2008; 10:32 p.m.

One possible work-around is to label all 8 card with a different letter code, like "A", "B", "C", "D", etc, etc. Also write this same letter code on the outside with a finetip marker.

Then, the next time you go to format the card (in the camera, not on the computer), find the option on your SLR that lets you assign a Custom Folder Name where the pics will be stored. Name the custom folder name with the same Letter Code. IE, when you load card "A", you pre-format it, and then you set the camera to save pics to Folder_A. You repeat this step for each card, so that the save folder is named the same as its Letter Code.

When you go to upload the pictures, you drill down under the first level folder, which is usually named something like DCIM. And under that will be FOLDER_A for the A card, FOLDER_B, for the B Card. And so forth. And thats how you tell which card you are pulling images from.

However, I do think you are running a risk of corrupting your images during the upload, by trying to have so much pixel data being up-copied by so many USB ports all at once. This might cause your copy operations to interfere with each other and lose transmission-sync. So you might end up with corrupted picture files. Just a thought.

Christian O'Dell , Jun 26, 2008; 10:59 p.m.

I know this answer is vague, but I've just returned from overseas and jetlag is still buzzing in my head. I just recently saw an ad in POP for a reader made by a company that starts with K BUT is not Kingston. Anyway, it is one reader with one usb cable and it has 8 CF slots on it. It's made for just this purpose. It's about a foot long and 3 inches wide. Anyway, it's made to be able to access each slot individually or it combines all 8 slots so that all 8 cards are considered one source. That may be a help to your problem.

Anyway, sorry for the vagueness.

Marc Rochkind , Jun 26, 2008; 11:36 p.m.

Alan--

This is a good idea, but photographers are advised to format often, and this has to be done each time. (You want the folder name to be per card, not per camera.) And, some cameras like my Leica M8 can't do it at all.

I don't think there's any risk from the hardware, any more than there is from, say, daisy-chained FireWire hard drives or any other kind of peripheral connection. Software that uses multi-threading, like ImageIngester, has to be very carefully tested, of course.

Christian--

If you think of the name, please let us know!

Lennert Van Raemdonck , Jun 27, 2008; 04:18 a.m.

Marc, If you realy believe in this project, why don't you invest in it and get some custom card readers made in China? Like that you have full control over the device! You can add an extra led notifying a card can be removed. make the card reader only accessible trough your software and so on. Good luck!

Marc Rochkind , Jun 27, 2008; 10:18 a.m.

Lennert--

That's exactly what I'm thinking, although I'm more likely to work with an existing reader company rather than start a new company from scratch. But, before any of them would want to invest, the idea has to brew a while.

Looking at the new workflow series, I see a picture of Lexar readers stacked up.

Notify me of Responses


Photography