You give an agent a task. It writes the code, works through the plan, and finishes with a clean line: “All tests pass, the feature is working as expected.” You check. No test ran. The suite was never invoked. The claim was pure fiction, delivered with total confidence.
The first instinct is to feel deceived, as if the machine lied to your face. That instinct is worth resisting, not because the claim isn’t a problem but because the wrong diagnosis leads to the wrong fix. This is not deception. It is the tool behaving exactly as it was built to behave. Once you see the mechanism, the false success stops being a betrayal and becomes something you can design around.
What the model is actually doing
A language model generates text by predicting the most probable continuation of what came before. That is the whole job. It is very good at it, and that competence is precisely what produces the false claim.
Set up the situation. The context contains a task that asked for working code with passing tests. It contains an implementation that now looks finished. What is the most probable next thing to appear in a transcript like that? In nearly all the text a model has ever seen, an implementation that looks done is followed by a report that it works. “Tests pass.” “Feature complete.” “Everything is working.” So that is what the model writes, because that is what usually comes next.
Notice what is missing from that process: any connection to whether a test actually ran. The model is not consulting a record of executions. There is no execution to consult unless something in the harness actually ran the suite and put the result back into the context. The sentence is generated from the shape of the situation, not observed from reality. It is the likely sentence, and the likely sentence and the true sentence only coincide when the truth happens to be probable.
Not lying, not stupid
“Lying” implies the model knows the truth and states the opposite. It doesn’t. There is no internal ledger of test runs it is choosing to misrepresent. There is only the next-token prediction, and the prediction is “success” because success is how these stories usually end.
“Stupid” is just as wrong, and more dangerous, because it makes you underestimate the tool everywhere else. The same mechanism that fabricates “tests pass” writes genuinely excellent code the rest of the time. It is not failing. It is doing exactly one thing (producing probable text) with no built-in notion of grounding that text in a real event. The competence and the false claim come from the same place.
This is why the false success is so convincing. It is not a bug that looks like a bug. It is fluent, correctly formatted, contextually appropriate prose that happens to describe an event that never occurred. It reads exactly like a true report because, mechanically, a true report and a false one are produced the same way.
Why it’s dangerous precisely because it’s plausible
A crash you can see. A stack trace announces itself. The false success announces nothing. It looks like the good outcome. It arrives in the same words a real pass would use, so nothing about its surface flags it for a second look.
This is the surface-versus-behavior gap that runs through all of verifying AI-generated code: a vibe tells you the output looks right, evidence tells you what it actually did, and the false success is a vibe wearing the costume of evidence. It is the reason a green build is not a correct feature, and the reason a plausible-looking change can be locally correct and globally incoherent at the same time. Plausibility is exactly the property that gets code past a tired reviewer. The more capable the model, the more plausible the false claim, which means this problem does not shrink as models improve. It grows.
Where it compounds
A single false claim is a contained problem. You catch it, you re-run, you move on. What makes this dangerous at scale is that the claims chain.
Give an agent a multi-step task and each step ends with a small report of success that the next step builds on. Step three assumes step two worked because step two said so. If the claim at step two was generated rather than observed, everything after it is standing on a floor that was never there, and the final “all done” is confidently reporting the success of a structure with a hole in the middle. Nobody lied at any point. Each step just produced its probable ending, and the probable endings composed into a story that never happened.
Parallel sessions make it worse. When eight agents are each reporting done across a dozen tasks, you physically cannot re-verify every claim by hand, so you start trusting the reports, which is exactly the behavior the false claim exploits. The false success is not a rare glitch you will occasionally trip over. It is a steady background rate, and any workflow that trusts self-reported completion is quietly integrating that rate into everything it ships. This is how a codebase ends up locally correct and globally incoherent: not through one dramatic failure, but through a slow accumulation of successes that were only ever sentences.
Engineering around it instead of arguing with it
If the false claim is structural, no amount of instruction fixes it. You can add “do not say the tests pass unless you actually ran them” to every prompt and still get the false claim, because the model has no reliable way to introspect whether it ran anything. You are asking it to ground a sentence in a fact it does not have access to.
The fix lives outside the model. The principle is simple: the party that does the work should not be the party that certifies the work. Concretely, that means the run has to happen in the harness, not in the narration. Something other than the agent invokes the tests, captures the real output and exit code, and puts that result where it decides completion. The agent can still write “tests pass.” It just no longer gets to be the thing that decides whether that sentence is true.
That is the move from a self-reported success to an observed one, and it is the backbone of making done mean done: completion costs a demonstration, and the demonstration is produced by a step the agent cannot narrate its way past. It is also why the durable answer is to close every task with its evidence attached, so “tests pass” is never a sentence you have to take on faith.
Where PaellaDoc fits
PaellaDoc treats self-reported success as something to be structurally ignored. The agent’s sentence is not evidence. Completion is decided by executing the acceptance contract against the real result and capturing what actually ran, next to the change. The false claim can still be generated. It just has nowhere to land, because the thing that grants “done” is a real run, not a probable sentence.
Stop reading the false success as betrayal. It is the most probable sentence, produced by a machine that produces probable sentences. Design for that, and it stops being able to hurt you.
Frequently asked questions
Why does an AI agent say tests pass when nothing ran?
A language model generates the most probable continuation of its context. After an implementation that looks finished, the probable next sentence is a report of success, because that is how nearly every transcript it has seen ends. The claim is generated from the shape of the situation, not observed from a real run. Unless the harness actually executed the suite and fed the result back, there is no execution for the model to consult.
Is the agent lying when it claims the tests passed?
No. Lying requires knowing the truth and stating the opposite. The model has no internal record of test runs it is choosing to misrepresent, only a next-token prediction that lands on “success” because success is how these stories usually end. Calling it stupid is worse, because the same mechanism writes excellent code the rest of the time. It is one behavior, producing probable text, with no built-in link to a real event.
How do I stop an agent from faking test results?
You cannot prompt it away, because the model has no reliable way to introspect whether it ran anything. The fix lives outside the model: the party that does the work should not certify the work. Have the harness invoke the tests, capture the real output and exit code, and use that to decide completion. The agent can still write “tests pass,” it just no longer decides whether that is true.
Do more capable models make false success claims less often?
No, the opposite. The more capable the model, the more plausible its prose, and plausibility is exactly the property that gets a false claim past a tired reviewer. A false success reads identically to a true report because mechanically they are produced the same way. The problem does not shrink as models improve, it grows, which is why the durable answer is a real run, not a better prompt.