For thirty years, the dominant hash functions all worked the same way inside. MD5, SHA-1, and the SHA-2 family (SHA-256, SHA-512) are built on the Merkle-Damgard construction: take a fixed-size compression function, chain it across message blocks, and carry a running state from one block to the next. The final state is the digest. It is a clean design, and it powered nearly all of digital signing, certificate chains, and integrity checking from the 1990s onward.
Then the cracks appeared. Xiaoyun Wang's team published practical collisions for MD5 in 2004 and a theoretical break of SHA-1 not long after. The SHAttered attack in 2017 produced two real PDF files with the same SHA-1 digest. None of this touched SHA-2 directly, which remains unbroken today. But NIST drew the right lesson anyway: if every deployed hash function shares one structural idea, a future attack on that idea would leave no fallback. So in 2007 they opened a public competition for a hash function built on something else.
The Competition and the Winner
Sixty-four submissions came in. After five years of public cryptanalysis, NIST selected Keccak in 2012, designed by Guido Bertoni, Joan Daemen, Michael Peeters, and Gilles Van Assche. Daemen was already co-author of AES, so the pedigree was serious. Keccak became the standard SHA-3 in FIPS 202, published in 2015.
The crucial point is what Keccak deliberately is not. It does not chain a compression function across blocks. It does not carry a small fixed state that an attacker can reason about as a clean intermediate value. It uses a sponge.
What a Sponge Actually Does
Picture a large internal state, in standard SHA-3 it is 1600 bits, split conceptually into two regions: the rate (the part the outside world touches) and the capacity (the part it never touches directly). The whole thing runs in two phases.
In the absorbing phase, the message is chopped into pieces the size of the rate. Each piece is mixed into the rate portion of the state, and then the entire state is scrambled by a fixed permutation called Keccak-f. Absorb a block, permute. Absorb the next, permute. The capacity bits are never written by the message and never read out during this phase. They just sit there, stirred by the permutation, accumulating entropy the attacker cannot see or set.
In the squeezing phase, you read output from the rate portion, permute, read again, and repeat until you have as many output bits as you want. For SHA3-256 you only need one squeeze. For an extendable-output function you can keep squeezing forever.
A sponge soaks up data (absorb) and then you wring output out of it (squeeze). The security comes from the capacity: a region of internal state that the message never gets to touch and the attacker never gets to observe. The bigger the capacity, the harder the function is to attack, and the smaller the rate, so the slower it runs. That single dial trades speed against security.
The security level of a sponge is roughly half its capacity. SHA3-256 uses a capacity of 512 bits, giving 256-bit security, and a rate of 1088 bits per absorbed block. SHA3-512 doubles the capacity to 1024, halves the rate, and runs slower as a result. The trade is explicit and easy to reason about, which is part of the design's appeal.
Inside Keccak-f: Five Steps That Repeat
The permutation that stirs the state is where the actual mixing happens. Keccak-f arranges the 1600 bits as a three-dimensional 5 by 5 by 64 array of bits and runs 24 rounds. Each round applies five steps with Greek-letter names, and you do not need to memorize the bit math to understand their jobs:
- Theta mixes each bit with the parity of nearby columns, spreading local changes across the state.
- Rho rotates bits within each lane by fixed offsets, so changes move along a different axis.
- Pi permutes the positions of the lanes, shuffling where things live.
- Chi is the only nonlinear step. It is what makes the function not solvable as a system of linear equations, and it is the heart of the cryptographic strength.
- Iota adds a round constant so that the rounds are not all identical, breaking symmetry that an attacker could otherwise exploit.
Diffusion (theta, rho, pi) and confusion (chi) alternate, which is the same Shannon principle that underlies block ciphers like AES. The difference is that here the permutation operates on a state far larger than any single message block, and the message only ever influences part of it.
The Length-Extension Problem It Quietly Solves
Here is the practical payoff that matters most in real systems. Merkle-Damgard hashes have a structural weakness called length extension. Because the digest is literally the final internal state, an attacker who knows SHA256(secret || message) and the length of the input can compute SHA256(secret || message || padding || extra) without ever knowing the secret. They just resume the hash from the published digest.
This is not academic. It is exactly why you cannot build a secure message authentication code by simply prepending a key to your data and hashing it with SHA-256. The fix the industry adopted was HMAC, a nested construction specifically designed to neutralize length extension on Merkle-Damgard hashes. HMAC works, but it exists because the underlying hash had a sharp edge.
Because a sponge never reveals the capacity bits, the published digest is not the full internal state. An attacker cannot resume the computation, because they are missing the half of the state that was never output. Length extension simply does not apply.
That means with SHA-3 you can build a keyed MAC by prefixing the key and hashing, the naive thing that is dangerous with SHA-2. NIST standardized exactly this as KMAC in SP 800-185. The sharp edge is gone, not patched.
SHAKE and the Extendable-Output Trick
The sponge has another property the old design could not offer cleanly. Since output comes from repeated squeezing, you can produce a digest of any length you like. NIST published these as SHAKE128 and SHAKE256, the extendable-output functions, or XOFs.
This turns out to be quietly important for modern cryptography. Post-quantum schemes lean on it heavily. The lattice-based and hash-based signature schemes now being standardized use SHAKE to expand seeds into large pseudorandom matrices and to derive arbitrary amounts of keying material from a fixed input. A single primitive that emits exactly as many bytes as you ask for removes a whole category of awkward truncation and re-hashing logic.
So Should You Switch From SHA-2?
No, and this is where honesty matters more than novelty. SHA-2 is not broken. It has held up to decades of cryptanalysis, it is faster in software on most CPUs without dedicated instructions, and it is the right default for the overwhelming majority of systems today. SHA-3 was never meant to replace SHA-2 because of a flaw, because there is no flaw to flee.
| Property | SHA-2 (Merkle-Damgard) | SHA-3 (Sponge) |
|---|---|---|
| Internal structure | Chained compression function | Sponge over a large permutation |
| Length-extension safe | No (needs HMAC) | Yes, by construction |
| Variable-length output | No | Yes (SHAKE) |
| Software speed (no HW accel) | Generally faster | Generally slower |
| Status | Unbroken, recommended | Unbroken, recommended |
The right way to think about SHA-3 is as cryptographic agility realized: a fully vetted, structurally independent fallback that is ready the day SHA-2 ever shows weakness, plus a cleaner toolkit (no length-extension footgun, native XOFs) for the systems being designed now. It is the backup you build before you need it, which is the only time you can build it calmly.
Where This Touches Real Software
Hash functions are load-bearing in places most people never see: they fingerprint the leaves of a Merkle tree, they derive keys through functions like HKDF, they authenticate messages, and they bind together the certificate chains your browser validates on every connection. Choosing the wrong construction for the wrong job, or assuming a bare hash is a safe MAC, is a recurring source of real vulnerabilities.
At Haven, the hashing primitives underneath our encryption are vetted, standard, and chosen for the specific job rather than for marketing. We would rather use a boring, decades-tested function correctly than a fashionable one carelessly. The sponge is a genuinely elegant piece of engineering, and the most useful thing it teaches is that a hash is never just a black box that turns data into a fingerprint. The shape inside determines what it is safe to build on top.