Как найти файл по md5

The other solutions are good but I want to propose one with fewer spawned processes, which should be significantly faster for many small files, if you have GNU find:

find /path/to/tree -type f -exec md5sum {} + | sed -nre 's/^md5-to-search-for  //p'

or without GNU find:

find /path/to/tree -type f -print0 | xargs -r -0 -- md5sum | sed -nre 's/^md5-to-search-for  //p'

answered Oct 3, 2013 at 8:41

Is there a way I can have a hash value as input when searching for files and a complete list of files and their locations as output?

This could be helpful when trying to pin point file duplicates. I often times find myself in situations where I have a bunch of files that I know I already have stored in some location but I don’t know where. They are essentially duplicates.

For instance, I could have a bunch of files on a portable hard drive, and also hard copies of those files on the internal hard drive of a desktop computer… but I’m not sure of the location! Now if the files are not renamed, I could do a file name search to try to locate the hard copy on the desktop. I could then compare them side by side and in case they are the same I could delete the copy I have on the portable hard drive. But if the files have been renamed on either one of the hard drives this would probably not work (depending on how much the new names differ from the original).

If a file is renamed, but not edited, I could calculate its hash value, e.g. SHA1 value is 74e7432df4a66f246b5214d60b190b67e2f6ce52. I would then like to have this value as input when searching for files and have the operating system search through a given directory or the entire file system for files with this exact SHA1 hash value and output a complete list of locations where these files are stored.

I’m using Windows, but I am generally interested in knowing how something like this could be achieved, regardless of operating system.

asked Dec 24, 2013 at 12:52

This is an intriguing question. I have been using a tool called fdupes to accomplish something similar. Fdupes will recursively search through directories and compare every file with every other file. First it compares size, and if the sizes are identical then it creates hashes of the files and compares that, if the hashes are the same then in actually goes through each file byte by byte and compares it.

When if finds all the files that are truly identical you can have it do several things. I have it delete the duplicate and create a hardlink in it’s place (thus saving me HDD space), although you can have it simply output the locations of the duplicate files and not do anything with them. This is the scenario you are asking about.

Some downsides with fdupes are that as far as I know it’s Linux only, and since it compares every file to every other file it takes quite a bit of I/O and time to run. It does not “search” for a file per say, but it would list all the files that have an identical hash.

I would highly recommend it and I set it to run in a cron job every day so that I never have any unnecessary duplicates of my data (it excludes my backups of course).

Fdupes Source Page

answered Dec 24, 2013 at 20:38

If you have PowerShell v.4.0 or higher, you can use the command:

Get-ChildItem _search_location_ -Recurse | Get-FileHash | 
Where-Object hash -eq (Get-FileHash _search_file_).hash | Select path

Where _search_location_ is folder or disk where you want to search for a duplicate and _search_file_ is a file that has a duplicate somewhere. You can put this command in a loop to search for several files or add | Remove-Item at the end of the line to automatically delete duplicates.

Also note that this command is suitable for small search folders only – it will take a lot of time if your search location has thousands of files (like a whole HDD).

There’s a tool ($) called FileLocator Pro that can search by file hash (SHA-x or MD5).

Excerpt from this page:
http://www.mythicsoft.com/filelocatorpro/help/en/advanced_criteria.htm

Note: If the expression type is set to ‘File Hash’ then the containing
text box can include a comma separated list of hash values or a
pointer to a file containing a list of hash values, e.g.

5A9C9B42A16F5E1985B7B0A019114C7A,675C9B42A16F5E1985B7B0A019114C7A

or,

=c:FileHashTable.txt

The actual algorithms used to calculate the hash, e.g. SHA1, MD5, are
specified in the Options tab.

answered Dec 30, 2013 at 13:42

Here’s an example for an MD5 algorithm:

Get-ChildItem "_search_location_" -Recurse | Get-FileHash -Algorithm MD5 | Where-Object hash -eq _hash_here_ | Select path

Replace _search_location_ with what directory you wish to search and replace _hash_here_ with the hash of the file you wish to find.

If you want to search for a hash besides the sha256 hash you add -Algorithm _algorithm_ after Get-FileHash where _algorithm_ is the chosen algorithm.

Beware that this requires PowerShell 4.0 and will recalculate every hash for every file for every search!

Voidtools Everything 1.5 search tool has an option to add a column of various hashes, such as CRC-32, CRC-64, MD5, SHA-1, SHA-256 for each file.

You can then search for a particular hash as well, for example md5:71E..

answered Apr 22 at 19:53

This is a good approach, but the search will be a lot faster if you only calculate hashes of files that have the right size. Using GNU/BusyBox utilities:

wanted_size=$(stat -c %s d1/x1)
wanted_hash=$(sha256sum <d1/x1)
find d2 -type f -size "${wanted_size}c" -execdir sh -c 'test "$(sha256sum <"$0")" = "$1"' {} "$wanted_hash" ; -print

answered Jan 25, 2017 at 2:06

To find a file with a known MD5 digest in a directory (using bash or ksh93):

x1digest="$( md5sum d1/x1 | cut -d ' ' -f 1 )"

for x2 in d2/*; do
    if [[ ! -f "$fpath" ]]; then
      continue
    fi

    x2digest="$( md5sum "$fpath" | cut -d ' ' -f 1 )"

    if [[ "$x2digest" == "$x1digest" ]]; then
      printf '"%s" has the same hash as d1/x1n' "$fpath"
    fi
done

On BSD systems, md5sum filename | cut -d ' ' -f 1 can be replaced by md5 -q filename.

answered Jan 23, 2017 at 10:28