TR Forums

Does anyone have a recommended application to scan multiple drives and find duplicate files?
Bonus points for something which presents the data in a way which shows entire folders which are the same.

I've used DFL in the past (DOS) and duplic8 for about 10 years but I need something more powerful.

We've been using Beyond Compare for a couple of years at work, and it works fabulously.

http://www.scootersoftware.com/

drsauced wrote:

We've been using Beyond Compare for a couple of years at work, and it works fabulously.

http://www.scootersoftware.com/

Do you know if Beyond Compare can compare not only the file itself, but also its metadata?

Ah, no it doesn't seem to be able to compare metadata for pictures. That'd be pretty useful...

I'll send them an email tomorrow and see what they say.

Beyond Compare looks like the Rolls Royce of duplicate file finders. Personally I am too cheap to pay for software like that, as I probably will only use it once or twice.

Here are some free alternatives, but you get what you paid for.

Auslogics Duplicate File Finder

CloneSpy

Duplicate Cleaner

Duplicate Files Finder

AllDUP

Tested clonespy, awful.
Working on dupe files finder and dupe cleaner now

AbRASiON wrote:

Does anyone have a recommended application to scan multiple drives and find duplicate files?
Bonus points for something which presents the data in a way which shows entire folders which are the same.

I've used DFL in the past (DOS) and duplic8 for about 10 years but I need something more powerful.

Sorry to hijack the thread but i was wondering how would you get so many duplicate files that would determine you to use a program to clean them up?

Here is something I used in the past: http://www.araxis.com/merge/index.html They offer a free 30 day trial.

What system are you running and what file attributes are you looking to compare? This could be a pretty simple script depending on what you're looking for

Just wanted to drop this in here. I whipped this up in Powershell (guessing you are using Windows). I created 2 directories with random files in them, then created similar folders in the directories (some the same and some different) and then moved files into those directories. Here's what I came up with



Pretty straightforward really. What this does is builds an array from whatever directories you chose (in this case 2 folders called test1 and test2 in C:), goes the through those folders and any subfolders it finds, then compares the second directory against the first. This will find any files that are the same, regardless of folder, and put them into a list formated by directory. If you want to output it to a file, simply add this to the end



and you will get a file in C: named results.txt that will have the results saved. It's quick and free. Best of all you can modify it however you'd like and achieve different results (try removing the -includeequal -excludedifferent and -passthru). Hope it helps!

What I would do is run a hashing algorithm - MD5 or SHA1, either is good - across everything. I'd then run a compare on all files with matching hashes. No matching hash means no duplicates - guaranteed. Matching hash could just be a collision (albeit pretty unlikely), hence the full compare.

Won't pick up on pictures that are identical but for metadata (or that are just resized versions), but it'll catch just about everything else.

I like the Duplicate File Finder from Rashid Hoda, http://rashid-hoda.software.informer.com/ That one has always worked well for me and you can do comparisons on combinations of name, binary contents, size plus the ability to choose directories and exclude directories.

If you haven't seen this already, here is a good list of free duplicate file finders: http://www.techsupportalert.com/best-fr ... tector.htm