Skip to content
Back to all field notes verify · 8 min

The trust gap: why output stopped being the bottleneck

The code arrives in seconds. Trusting it takes the rest of your afternoon. The bottleneck moved from producing software to believing the software you produced.

field note 8 min

The code shows up almost instantly now. You describe a feature, an agent writes it, and thirty seconds later there is a diff on your screen that looks plausible. Then you spend the next forty minutes deciding whether to believe it. Reading it, poking at it, half-trusting it, shipping it with a knot in your stomach. The writing was free. The trusting cost you the afternoon.

That gap, between code existing and code you are willing to put your name on, is the actual bottleneck now. It was always there. It used to be hidden inside the time it took to write the code in the first place, because if you wrote it slowly by hand, you trusted it as a byproduct of building it. Remove the slow writing and the trust doesn’t come free anymore. It stands there on its own, unpaid for, and it turns out to be the expensive part.

Output was never really the constraint

We spent decades optimizing production. Faster languages, better editors, autocomplete, frameworks, snippets, all aimed at getting code out of your head and into the file faster. It worked, and the whole time it was solving the wrong problem, or rather a problem that was about to disappear.

Because production is now solved. Not perfectly, but well past the point where it constrains anything. The models write code. The bottleneck did not vanish when production got cheap, it moved, the way a bottleneck always moves to the next scarcest thing. And the next scarcest thing is trust. You can generate ten implementations of a feature before lunch. You cannot trust ten implementations before lunch. The reading, the checking, the “does this break the thing next to it,” that did not speed up at all. If anything it got slower, because now you are vouching for code you did not write and do not fully understand.

This is why “the AI made me faster” so often feels like a lie by dinnertime. Generation got faster. The afternoon did not get shorter. The time just relocated, from writing to verifying, and nobody warned you that the second one is harder than the first.

Trust is not a feeling, and that’s the good news

Here is the trap. Because trust feels like a subjective state, we try to earn it subjectively. We read the diff harder. We stare at the code until the discomfort fades and call that confidence. But a feeling of confidence is not the same as a code that is correct, and the feeling is exactly the thing a fluent, confident-sounding agent is best at manufacturing. An agent that writes clean-looking code and reports “done, tests pass” produces the sensation of trustworthiness whether or not the code deserves it.

So subjective trust does not scale and does not even track reality well. The escape is to stop trusting the code and start trusting the process that produced the evidence. That is a different object entirely. You do not have to believe the agent’s sentence. You have to be able to check the claim behind it, mechanically, the same way every time.

Mechanical trust looks like this. There is a specification of what correct means, written before the code, so “correct” is not decided by looking at whatever the agent produced. There is a check that the agent did not write and cannot edit, and it passes or fails on its own. There is a record of the check running, real output, not a summary. When those three exist, your trust is not in the model’s word. It is in a chain of evidence you can inspect. That kind of trust scales, because verifying a claim against fixed evidence is fast, and it holds, because the evidence does not change its story to please you.

How the gap actually closes

You close the trust gap by shrinking the part that depends on you personally reading everything, and growing the part that a machine can vouch for.

Move the definition of correct before generation. If you decide what “working” means only after seeing the code, the code has already framed your judgment, and your trust is really just familiarity. Acceptance criteria set beforehand give you something to check against that the implementation did not get to shape.

Make the evidence a real artifact, not a claim. “Tests pass” is a sentence. A test process that exited zero, with its output captured, is evidence. The difference is the whole game. One you have to believe, the other you can look at. Closing work with the proof attached is what turns a promise into a fact you can act on without re-reading the world.

Verify the boundaries the agent is blind to. A model writes each piece to be locally correct, and local correctness is exactly what it is good at. The trust gap lives at the seams, where the new code meets the old, where an assumption here contradicts an invariant there. This is the locally correct, globally incoherent problem, and it is precisely the region where reading the diff tells you least, because the diff shows you the change and hides everything the change was supposed to respect. Trust here comes from checks that span the seams, integration tests and contracts, not from staring harder at one file.

Reserve your attention for what only you can judge. Mechanical trust does not remove the human, it repositions them. The machine vouches for “the tests pass and nothing regressed.” You vouch for “this is the right behavior to build.” Those are different jobs, and collapsing them is what burns your afternoon. When the mechanical layer is solid, your reading time goes to the decisions that actually need a person, not to babysitting a green checkmark.

The gap is where the discipline lives now

There is a whole shift hiding in this, from producing software to assuring it, and it reorders the work. When output was scarce, the good engineer was the one who could produce. When trust is scarce, the good engineer is the one who can establish it, quickly and repeatably, over a volume of code no human could read line by line. That is not a downgrade of the craft. It is the craft moving to where the constraint moved.

None of it means reading less carefully when reading matters. It means not spending your one scarce resource, careful attention, on things a machine should have vouched for before the code ever reached you. The trust gap is the real subject of verifying AI-generated code: not how to make the agent produce more, but how to make its output believable at the speed it arrives.

Where PaellaDoc fits

The trust gap is exactly the seam PaellaDoc is built to hold. The criteria you set before generation live next to the code, so “correct” is not something you reconstruct from the diff. The checks run and leave a record, so the evidence is an artifact you can inspect instead of a sentence you have to believe. And the work does not reach “done” until that evidence exists. You stop being the entire trust mechanism, reading every line to feel okay about shipping, and go back to spending your judgment where it counts: deciding what should be true, and letting the system prove that it is.

Frequently asked questions

What is the trust gap in AI-generated code?

It is the distance between code existing and code you are willing to put your name on. When you wrote code slowly by hand, you trusted it as a byproduct of building it. Agents removed the slow writing, and the trust no longer comes free. It stands there unpaid for, and it turns out to be the expensive part: the code arrives in seconds, believing it takes the rest of the afternoon.

Why doesn’t AI actually make me faster overall?

Because generation got faster while verification did not. You can produce ten implementations of a feature before lunch, but you cannot trust ten before lunch. The bottleneck did not vanish when production got cheap, it moved to the next scarcest thing, which is trust. The time relocated from writing to verifying, and verifying is harder, because now you are vouching for code you did not write and do not fully understand.

How do I trust code I didn’t write and don’t fully understand?

Stop trying to earn trust subjectively by reading the diff harder, because that only manufactures a feeling a fluent agent is good at producing. Trust the process instead: a specification of correct written before the code, a check the agent cannot edit that passes or fails on its own, and a recorded run with real output. That chain of evidence is inspectable and it scales, because verifying a claim against fixed evidence is fast.

Where does AI-generated code most often go wrong?

At the seams, where new code meets old, where an assumption here contradicts an invariant there. A model writes each piece to be locally correct, which is exactly what it is good at, so the diff looks clean while the incoherence hides in what the change was supposed to respect. Reading the changed file tells you least here. Trust at the boundaries comes from integration tests and contracts that span the seams, not from staring harder at one file.