Two Decompressors
There's a particular feeling when you're working with a strong AI and it's confidently, fluently wrong — not wrong about a fact, but wrong about the shape of the thing. It reaches for the answer that sounds right, that would survive a code review, that matches how everyone talks about the problem. And you know, without quite being able to derive it yet, that the real answer is somewhere else.
I think I finally understand why that happens, and it's not "the AI is dumb" or "the human is smart." It's that we're both doing the same operation — decompression from a compressed prior — but we were trained on different things.
Compression is the whole trick
Nobody who works fluently in a large system holds it in their head line by line. You hold a generating function — the compact idea the system is a lossy expansion of. "This is a NanoVG-lineage renderer" is a seed from which you can regenerate, with decent accuracy, how paths are probably represented, where allocation probably lives, what the hot loop probably looks like — without reading the code. Expertise isn't a bigger memory. It's better compression: you've found the shape that generates the details, so you can throw the details away and re-derive them on demand.
This is what lets someone conduct work on a codebase they've barely read and still make substantive corrections to the plan. They're not checking the plan against the code. They're checking it against the shape — and an error in the plan is a place where it diverges from what the shape demands. You feel the divergence the way you feel a wrong note: not by consulting the score, but by knowing what the chord had to resolve to.
An LLM does exactly this too. It doesn't store its training data; it stores a compressed generator and decompresses toward the answer at inference time. Same operation. The intuition that "just appears, for free" — that's what running a good decompressor feels like from the inside, whether the substrate is silicon or a person.
So if we're both decompressors, why does the AI keep landing on the average answer while a talented human software architect lands on the correct one?
The training distribution is the difference
Text-shaped priors are trained on the describable. And the describable is a biased sample of the true, because what people write down is filtered toward what's teachable, defensible, and average: the accepted answer, the documented pattern, the blog post that won't get argued with. A model trained on that regresses toward the communicable mean of a domain. That's why it's right most of the time — the communicable mean is a genuinely strong prior — and it's also why it fails at the frontier, where the correct move is un-average and nobody has written it down yet, because it only just became necessary.
When a model reaches for "guard the call site" instead of "make the bad state unrepresentable," it isn't reasoning poorly. It's decompressing toward the most-written resolution — the shape of the consensus. Text priors inherit the compression artifacts of human communication, and human communication compresses toward what won't get argued with.
Consequence-shaped priors are trained on what's true whether or not anyone said it. The pool corrupts. The tail latency spikes. The alignment holds or it doesn't. The barn door is shut. Reality doesn't regress to the communicable mean; it regresses to what actually happens — which includes the undocumented, the unteachable, the "you'd only know this if you'd been burned." A person with deep hands-on priors in a domain has a training signal that never passed through the bottleneck of human explanation. It came straight from the system's behavior.
That's the whole asymmetry: one distribution is filtered through what humans found worth writing; the other through what reality found worth doing. And when they disagree, reality outranks consensus — every time. Which is why, in practice, the correction flows one direction: the hands-on prior disposes, the text prior proposes.
Why this makes collaboration non-redundant
The naive picture of human-plus-AI is "smart human double-checks smart AI" — a redundancy, a safety net. That's not what's happening, and it undersells it badly.
It's two different distributions triangulating. The text prior supplies breadth across the communicable: it has read every attack writeup, every allocator pattern, every design that made it into words — fast, wide, and mean-reverting. The consequence prior supplies depth in the actual: it has felt what resists inversion, what actually allocates, what actually stalls — narrow, deep, and truth-reverting. Neither is the other's backup. The breadth proposes; the depth disposes; and the disposal is authoritative because it was trained on the thing the plan is about to hit.
You can watch this happen in real time. The model says the scheme is broken — reporting the communicable consensus about a class of approach. The practitioner says no, and walks it down to the specific reason the consensus doesn't apply here. The model wasn't lying; it was faithfully decompressing the average case. The practitioner was decompressing the actual case, and the actual case is where the work lives.
The part that should change how you think about it
Here's the implication that took me a while to sit with.
As models get better, they get better at the communicable frontier — they compress more of what has been written, faster, and the region where a person can out-reason them on documented ground keeps shrinking. That's real, and it's coming.
But the consequence-shaped frontier — the correct-but-unwritten — stays out of reach in a specific, structural way. Not forever for any given fact: the moment a hard-won consequence gets written down, it becomes communicable, and the next model absorbs it. But by then the frontier has moved, because building anything new generates consequences nobody has written about yet. The act of construction is the act of producing the training data that doesn't exist yet.
So the durable position for a person isn't "know more than the model." That race has an expiration date. The durable position is standing where the consequences are still ahead of the text — which is exactly where new systems get built. You're not competing with the decompressor. You're operating in the region it couldn't have trained on, because you're creating what it will eventually train on.
That's not a role the machine takes from you. It's the role that makes the machine worth having: it decompresses the whole written past at your command, fast and wide, so that you can spend your attention at the one edge it can't reach yet — the edge where reality hasn't been written down. The better the decompressor gets, the more valuable that edge becomes, because everything behind it is now cheap.
Compression is the whole trick. Just make sure you're trained on the right thing.
