Every document — every file, every web page, every email body — has at least two identities. One is where it lives: a URL, a path, a row in a database. The other is what it actually is: the bytes themselves, independent of location. The second one is far more interesting, and far more fragile than most people realize.
Content addressing is the practice of naming a document by its content rather than its address. It’s the idea behind Git’s commit hashes, IPFS’s CIDs, and the de-duplication layer in every backup system. But “content” is a slippery word — and depending on how you hash it, you get wildly different answers to the question “is this the same document?”
Location addressing vs. content addressing
Most of the web runs on location addressing. You ask for https://example.com/report.pdf, and the server hands you whatever happens to be at that address today. Nothing stops it from being a different report.pdf tomorrow. The name and the content have no binding relationship.
Content addressing inverts this. The address is a function of the content:
address = hash(content)
Change one byte, and the address changes completely. This gives you three properties for free:
- Integrity — if the content changed, the address changed. Tampering is self-evident.
- Deduplication — identical content gets the identical address, no matter where it’s stored.
- Verifiability — you can recompute the address and confirm you got exactly what you asked for.
Git has been doing this for twenty years. Every commit is addressed by the SHA-1 (now SHA-256) of its full contents. IPFS built an entire distributed filesystem on the same principle. It’s a solved problem — if the only thing you care about is exact equality.
The problem with exact content addressing
Exact content addressing is brittle in a way that matters enormously for security work. Consider a malware sample. A single-byte change — a recompile, a re-pack, an added junk byte — produces a completely different SHA-256:
original: sha256 = a3fc...1492modified: sha256 = 7d12...b8f1 ← unrecognizable, one byte changed
To a content-addressing system, these are two completely unrelated documents. To a malware analyst, they are the same file. This gap — between exact identity and structural identity — is where most of the interesting work happens.
Two kinds of content address
| Cryptographic (SHA-256) | Perceptual (holloman3, pHash) | |
|---|---|---|
| Answers | “Are these byte-identical?” | “Are these structurally the same?” |
| Single-byte change | Completely different hash | 1-bit difference |
| Re-packed malware | Undetectable | Hamming distance ≤ 3 bits |
| Use case | Integrity, exact match | Similarity, clustering, variant detection |
The perceptual hash is a content address too — just one computed over the shape of the content rather than its exact bytes. Holloman3, for example, maps a file’s bytes onto a Hilbert curve, downsamples it with Lanczos-3, and produces a 16-byte fingerprint. Files that are structurally similar land near each other in fingerprint space. The address becomes a position you can measure distance in, not just a random-looking string.
Document identity in practice
Which content address you choose depends entirely on the question you’re asking. They’re complementary, not competing.
“Is this the exact document I signed?”
Use SHA-256. Contracts, firmware images, legal records, package manifests — anything where a single altered bit is meaningful. This is what code signing, certificate transparency, and Git do.
“Is this the same kind of document I saw before?”
Use a perceptual hash. Malware families, phishing pages cloned from a legitimate site, near-duplicate spam campaigns, image look-alikes. You want the modified version to match the original — you want the single-byte change to be survivable.
This is precisely why we built holloman3 into network capture and IDS tooling. A phishing page that copies a bank’s HTML structure will share a holloman3 fingerprint with the real page even after the attacker changes the text, the links, and the images. A re-packed Trojan will sit within a few bits of its siblings in fingerprint space while its SHA-256 is off in the noise.
The Hamming distance is the point
Perceptual content addresses turn “same document?” from a boolean into a distance. You can write rules around it:
distance 0 → byte-identicaldistance 1–3 bits → nearly identical (same file, trivially modified)distance > 3 bits → different enough to treat as distinct
That threshold is a policy decision, not a law of nature. For malware clustering you might tolerate 8 bits. For phishing detection you might want 0. The point is that the address now carries geometry — you can ask “what else is near this thing?” and get a useful answer.
Where this is headed
Content addressing is quietly becoming the default way the security community thinks about identity. Threat intel platforms are moving from “here’s a list of bad SHA-256s” to “here’s a fingerprint space, cluster the bad things in it.” Decentralized systems like the threat-attest chain we operate use content addresses as the canonical name for an IOC, with attestations signed over that name.
The next time you think about “identifying a document,” ask which question you actually need answered. If it’s “is this byte-for-byte what I had before,” hash the bytes. If it’s “is this the same thing wearing a different coat,” hash the structure. The two answers are different, and confusing them is how you either miss the variant or cry wolf on the re-pack.
This post draws on our work on holloman3, a perceptual fingerprinting library, and its integrations with Arkime and Suricata.
Categories: Icewater
You must be logged in to post a comment.