Personal computing discussed

Moderators: morphine, Steel

 
AbRASiON
Gerbil Team Leader
Topic Author
Posts: 292
Joined: Tue Mar 18, 2003 8:06 pm

Recommended duplicate file finder? (binary file compare)

Sat Jul 16, 2011 11:34 pm

Does anyone have a recommended application to scan multiple drives and find duplicate files?
Bonus points for something which presents the data in a way which shows entire folders which are the same.

I've used DFL in the past (DOS) and duplic8 for about 10 years but I need something more powerful.
 
drsauced
Gerbil Jedi
Posts: 1535
Joined: Mon Apr 21, 2003 1:38 pm
Location: Here!

Re: Recommended duplicate file finder? (binary file compare)

Sun Jul 17, 2011 1:19 pm

We've been using Beyond Compare for a couple of years at work, and it works fabulously.

http://www.scootersoftware.com/
Calm seas never made a skilled mariner. But, sadly I'm an A's fan.
 
bdwilcox
Graphmaster Gerbil
Posts: 1261
Joined: Mon Apr 21, 2003 12:21 pm

Re: Recommended duplicate file finder? (binary file compare)

Sun Jul 17, 2011 3:43 pm

drsauced wrote:
We've been using Beyond Compare for a couple of years at work, and it works fabulously.

http://www.scootersoftware.com/

Do you know if Beyond Compare can compare not only the file itself, but also its metadata?
 
drsauced
Gerbil Jedi
Posts: 1535
Joined: Mon Apr 21, 2003 1:38 pm
Location: Here!

Re: Recommended duplicate file finder? (binary file compare)

Mon Jul 18, 2011 1:06 am

Ah, no it doesn't seem to be able to compare metadata for pictures. That'd be pretty useful...

I'll send them an email tomorrow and see what they say.
Calm seas never made a skilled mariner. But, sadly I'm an A's fan.
 
Dirge
Gerbil Jedi
Posts: 1608
Joined: Thu Feb 19, 2004 3:08 am
Location: New Zealand

Re: Recommended duplicate file finder? (binary file compare)

Tue Jul 19, 2011 2:16 am

Beyond Compare looks like the Rolls Royce of duplicate file finders. Personally I am too cheap to pay for software like that, as I probably will only use it once or twice.

Here are some free alternatives, but you get what you paid for.

Auslogics Duplicate File Finder

CloneSpy

Duplicate Cleaner

Duplicate Files Finder

AllDUP
FDISK /MBR
 
AbRASiON
Gerbil Team Leader
Topic Author
Posts: 292
Joined: Tue Mar 18, 2003 8:06 pm

Re: Recommended duplicate file finder? (binary file compare)

Mon Aug 08, 2011 6:48 am

Tested clonespy, awful.
Working on dupe files finder and dupe cleaner now
 
Arclight
Gerbil Elite
Posts: 750
Joined: Tue Feb 01, 2011 3:50 am

Re: Recommended duplicate file finder? (binary file compare)

Wed Jan 11, 2012 3:49 am

AbRASiON wrote:
Does anyone have a recommended application to scan multiple drives and find duplicate files?
Bonus points for something which presents the data in a way which shows entire folders which are the same.

I've used DFL in the past (DOS) and duplic8 for about 10 years but I need something more powerful.


Sorry to hijack the thread but i was wondering how would you get so many duplicate files that would determine you to use a program to clean them up?
nVidia video drivers FAIL, click for more info
Disclaimer: All answers and suggestions are provided by an enthusiastic amateur and are therefore without warranty either explicit or implicit. Basically you use my suggestions at your own risk.
 
bitcat70
Gerbil
Posts: 53
Joined: Fri Jan 28, 2011 3:02 pm

Re: Recommended duplicate file finder? (binary file compare)

Wed Feb 22, 2012 12:38 pm

Here is something I used in the past: http://www.araxis.com/merge/index.html They offer a free 30 day trial.
 
LaChupacabra
Gerbil First Class
Posts: 145
Joined: Tue Dec 30, 2008 10:59 pm

Re: Recommended duplicate file finder? (binary file compare)

Wed Feb 22, 2012 1:37 pm

What system are you running and what file attributes are you looking to compare? This could be a pretty simple script depending on what you're looking for
 
LaChupacabra
Gerbil First Class
Posts: 145
Joined: Tue Dec 30, 2008 10:59 pm

Re: Recommended duplicate file finder? (binary file compare)

Thu Feb 23, 2012 2:35 pm

Just wanted to drop this in here. I whipped this up in Powershell (guessing you are using Windows). I created 2 directories with random files in them, then created similar folders in the directories (some the same and some different) and then moved files into those directories. Here's what I came up with

$var1 = @(Get-ChildItem c:\test1 -recurse)
$var2 = @(Get-ChildItem c:\test2 -recurse)
Compare-Object $var1 $var2 -includeequal -excludedifferent -passthru


Pretty straightforward really. What this does is builds an array from whatever directories you chose (in this case 2 folders called test1 and test2 in C:), goes the through those folders and any subfolders it finds, then compares the second directory against the first. This will find any files that are the same, regardless of folder, and put them into a list formated by directory. If you want to output it to a file, simply add this to the end

$var1 = @(Get-ChildItem c:\test1 -recurse)
$var2 = @(Get-ChildItem c:\test2 -recurse)
Compare-Object $var1 $var2 -includeequal -excludedifferent -passthru > c:\results.txt


and you will get a file in C: named results.txt that will have the results saved. It's quick and free. Best of all you can modify it however you'd like and achieve different results (try removing the -includeequal -excludedifferent and -passthru). Hope it helps!
 
sjl
Gerbil
Posts: 71
Joined: Tue Dec 07, 2004 5:14 pm

Re: Recommended duplicate file finder? (binary file compare)

Sat Dec 08, 2012 6:55 am

What I would do is run a hashing algorithm - MD5 or SHA1, either is good - across everything. I'd then run a compare on all files with matching hashes. No matching hash means no duplicates - guaranteed. Matching hash could just be a collision (albeit pretty unlikely), hence the full compare.

Won't pick up on pictures that are identical but for metadata (or that are just resized versions), but it'll catch just about everything else.
 
nanoflower
Gerbil Team Leader
Posts: 270
Joined: Wed Mar 04, 2009 1:10 pm

Re: Recommended duplicate file finder? (binary file compare)

Sat Dec 08, 2012 8:07 am

I like the Duplicate File Finder from Rashid Hoda, http://rashid-hoda.software.informer.com/ That one has always worked well for me and you can do comparisons on combinations of name, binary contents, size plus the ability to choose directories and exclude directories.
 
kroker
Gerbil
Posts: 11
Joined: Sun Feb 21, 2010 3:26 am

Re: Recommended duplicate file finder? (binary file compare)

Fri Mar 22, 2013 3:53 am

If you haven't seen this already, here is a good list of free duplicate file finders: http://www.techsupportalert.com/best-fr ... tector.htm

Who is online

Users browsing this forum: No registered users and 2 guests