apkellogg wrote:Is there a program to find duplicate files across multiple hard drives? Basically, I have shared folder that too many people have had access to over the years and I would like to see if there are duplicate files saved at multiple places in the folder under different file names. I am using Windows XP Pro/MCE 2005.
Thanks you for any advice.
find "$@" -type f -print | sed -e 's/^/sha1sum "/; s/$/"/' | bash | sort | uniq --all-repeated=separate --check-chars=40 | sed -e "s/[^ ]* .//"
Flying Fox wrote:That doublekiller thing looks to be able to compare even by size and dates.
just brew it! wrote:I'm surprised that the tools give false positives; if the length of the files match, the tool should then do a byte-for-byte comparison of the contents to verify the match.
While false-positives are theoretically possible with a checksum-based approach like the one I gave the script for above, the odds are mathematically so low (it's a 160-bit hash, so the odds of getting a collision are vanishingly small) that practically speaking you'll never see one.