File Hashes for Integrity: MD5, SHA-1, SHA-256 and When HMAC Applies
A hash function compresses arbitrary input into a fixed-length fingerprint: change one bit of the file and the fingerprint changes completely. That property makes hashes the workhorse of integrity checking — comparing two files, verifying a download, or proving a backup did not drift — but the choice of function and the surrounding protocol matter more than most people assume.
Choosing the algorithm
- SHA-256 is the default for anything that matters: download verification, release manifests, forensic copies, backup manifests. Compute it everywhere with the same command shape:
sha256sum backup.tar.gz openssl dgst -sha256 -r image.iso
- SHA-1: collision-broken in practice since 2017 (the SHAttered attack produced two PDFs with equal SHA-1). It survives in legacy manifests and old VCS object IDs, but new integrity schemes should not use it.
- MD5: thoroughly broken for adversarial use — collisions are cheap to manufacture. Still acceptable for detecting accidental corruption (a truncated transfer, a bad sector), and still what many mirror sites publish. Treat it as a typo detector, not a security control.
The part hashes cannot do
A plain hash proves the file matches the hash you already had. It does not prove the hash itself is honest. If an attacker controls both the file and the published checksum — a compromised mirror, a man-in-the-middle on the download page — the hashes agree and the malware ships anyway. Robust download verification therefore needs the hash to come from a channel the attacker does not control: a signed release message, an HTTPS page on the vendor’s own domain, or a package manager’s signature infrastructure.
Where HMAC takes over
When you need both parties to verify integrity and authenticity with a shared secret, use HMAC (Keyed-Hash Message Authentication Code) rather than hashing the secret alongside the data yourself. HMAC is the correct construction — HMAC-SHA256 appears in webhook signature headers (including WhoDNS’s own Telegram relay callbacks), API request signing, and JWTs. The distinction in one line: a plain hash answers “is this the same file?”, an HMAC answers “did someone who knows the secret produce this?”. Never roll your own sha256(secret + data); length-extension attacks against that pattern are textbook material, which is exactly why HMAC exists.
Practical workflow
- Publish SHA-256 manifests with releases and commit them separately from the artifacts.
- Verify backups at write time and again after restore drills — the second check is the one that finds silent bit rot.
- For webhooks, validate HMAC with a constant-time comparison and never skip it because “the endpoint is unlisted”.
To fingerprint a file, a pasted string or a generated token without installing anything, the hash generator computes MD5, SHA-1, SHA-256 and friends in the browser — handy for the everyday cases where a quick, correct checksum is all you need.