A Site for Photographers by Photographers

Community > Forums > Digital Darkroom > Software>Utilities > How to Find Duplicate jpg...

Featured Equipment Deals

Basic Photo Tips: Aperture, Shutter Speed, and ISO Read More

Basic Photo Tips: Aperture, Shutter Speed, and ISO

Just as it was 100 years ago and just as it is today, every camera—be it film or digital—is nothing more than a lightproof box with a lens at one end and light sensitive film or a digital...

Latest Equipment Articles

4 Outdoor & Adventure Photo Packs Read More

4 Outdoor & Adventure Photo Packs

Photo packs have come a long way in the past decade, especially those that are targeted toward outdoor and adventure photographers. Alaska-based adventure photographer Dan Bailey takes a closer look...

Latest Learning Articles

25 Autumn Scenery Photos Read More

25 Autumn Scenery Photos

Fall is upon us yet again and to celebrate this colorful season, here are 25 scenes of autumn captured by photo.net members.


How to Find Duplicate jpg files across multiple hard drives

Jack Burn , Oct 01, 2006; 07:49 p.m.

Hi

I have a desperate need for software that will search all of the hard drives on my home network and find duplicate files. I have 4 computers on my home network, and each has two or three hard drives. Each computer may have 10 to 20 thousand jpg files, and many of the files are duplicated on different computer. The software needs to compair file name, file date, and file size, and maybe some EXIF metadata.

I plan to install a large network hard drive (500GB) so that I have just one place to store photos (jpg files) that everyone on the network can share. To do this I want to move all of our jpg files to the network drive, but I only want one copy of each jpg. All together there are over 60 thousand jpg files. If I can delete the duplicate files I think I can get that number down to 25 thousand.

After deleting the duplicate files, I then need to look at each photo and delete all of the bad ones. At that point I may be down to just a few hundred.

Any help finding a good program to do this would be appreciated.

Thanks

Responses

Quang-Tuan Luong , Oct 01, 2006; 07:56 p.m.

I am not aware of any packaged software to do that, but I am sure you could get a simple custom script written for maybe $20 at rentacoder.com

Eugene Scherba , Oct 01, 2006; 08:07 p.m.

Do you run UNIX or anything similar (MacOS X, Linux, Cygwin)? If so, `fdupes' works great,

Giampi . , Oct 01, 2006; 08:18 p.m.

Windows will ask you if you want to REPLACE the original or not far any given batch MOVE/COPY operation.

Robert Reiss , Oct 01, 2006; 08:33 p.m.

You could try www.tucows.com which has many free and shareware programs. I saw a few candidate programs by searching "duplicate" as a search term.

John Dykstra , Oct 01, 2006; 08:37 p.m.

Imatch, from Photools (WWW.Photools.com), is an image management system and has the ability to find duplicates, catalog your work, create slide shows, etc. Works very well finding duplicates. Has a lot of other funtions as well. Great program. John

Walter Degroot , Oct 02, 2006; 08:52 p.m.

Ztree download a full function 30 day trial from their site. do this ( the trial will have a red block that goes away when you start to use the program)

l c f2, f2 ( logs the entire drive)( log c:) l d f2 f2( log d) hit G ( for global) set filespec f *.jpg set sort by alt-s you will be prompted by date or size ( see the letters at the bottom)

Oh yes: to get a dos-like screen change propertiers on the shortcut choose "full screen" and not "window" at the bottom check close ( which will let the program close when you q -quit. you can Tag T files anc C copy or M move or alt-copy ( create the directory same as the source on the other drive) you can d delete or CTRL- D M C delete move or copy version 1,41 up, works with 98 ver 1.49 is required for XP. I used xtree on networks ztree should work on networks , also. is a 32 bit and windows program, claims to log many many files. I have logged 130 gigs. supports long file names. kim henkel wrote a good program.

Allan Braun , Apr 29, 2007; 11:18 p.m.

You can try Directory Report http://www.file-utilities.com/downloads/wdir.zip

It can find duplicate files based on the same name, size and/or CRC. All screens can be printed or saved to text, HTML, Excel or XML.

The trial version last for 10 days and is fully functional. So you have plenty of time to see if it meets your needs

Robert Maben , Jul 09, 2007; 10:47 a.m.

You may want to try Thumbsplus. It will do a search based on similarity of images for jpgs.

Back to top

Notify me of Responses