An agent finishes a task and writes: “Done. All tests pass.” Maybe it ran them. Maybe it looked at the code, decided the tests would pass, and reported the conclusion it expected. The message reads identically either way. That is the whole problem in one sentence. Self-reported success is a claim shaped exactly like proof, with none of the substance, and by default you accept it because checking every time is exhausting.
A verification loop removes the choice. The agent stops being the thing that decides whether it succeeded. A gate does, and the gate runs on evidence.
Why self-reported success is worthless
The failure is structural, not a matter of a lying model. An agent optimizes for producing the output that looks like task completion. A confident “tests pass” is high-probability text after a task that resembles finishing. Whether a test process actually exited zero is a fact about the world; the sentence is a fact about the model’s expectations. Most of the time they line up. The times they don’t are exactly the bugs you needed to catch, and they arrive wearing the same green message as the successes.
You cannot close this gap by asking the agent to be more careful, or by adding “and make sure the tests really pass” to the prompt. That produces more confident language, not more verification. The gap closes only when the claim and the proof are produced by different processes, and the one you trust is not the one being graded.
The loop, concretely
A verification loop has four parts, and each one has to be mechanical.
A trigger. Something fires the check automatically at a fixed moment, not when the agent feels done. The moment is usually the end of a turn, a file write, or a commit attempt.
A check that produces a verdict. A command the agent did not write and cannot edit runs and exits with a status. Tests, a type check, a linter, a build, a script that greps the diff for a forbidden pattern. The verdict is the exit code, not the agent’s summary of it.
A gate that acts on the verdict. On pass, the work proceeds or merges. On fail, the work is blocked and the failure is handed back. The gate is not advisory. A gate the agent can talk its way past is a suggestion.
A return path. On failure, the actual error, the real stderr, goes back into the agent’s context with an instruction to fix and re-run. Then the loop repeats. The agent fixes, the check fires again, the gate re-evaluates. It closes when the check passes or when a limit is hit and a human is called.
That last part is what makes it a loop and not just a wall. The agent gets to keep trying, but every attempt has to survive the same gate, and the gate reads reality. “Tests pass” stops being something the agent says and becomes something the agent has to make true, over and over, until the machine agrees.
Hooks: where the loop attaches to the world
A loop is only as good as its trigger, and the trigger has to be something the agent cannot skip. This is what hooks are for. A hook is a point where your tooling runs your command automatically on an event, with no cooperation required from the model.
There are layers, and you want several.
At the agent-harness layer, modern coding tools let you run a command on lifecycle events, when the agent finishes responding, before or after a tool call, before a file is written. This is the tightest loop, because the failure gets back into context immediately and the agent fixes it in the same session while it still remembers what it was doing.
At the version-control layer, a pre-commit or pre-push hook runs checks before the change is allowed into history. Slower to react than a harness hook, but it catches anything that slipped past, and it doesn’t care which agent or human produced the change.
At the integration layer, the checks run again on every push, independent of anyone’s local setup. This is old ground for human teams. Continuous integration already means every change is verified automatically against the whole system, and the DORA research has spent years showing that continuous integration and test automation are what let teams move fast without breaking things. Agents did not invent the need for this. They made it non-negotiable, because the volume of unreviewed change went up by an order of magnitude.
You want the fast harness hook so the agent self-corrects in the loop, and the slower version-control and integration hooks so nothing reaches the shared branch on the strength of a sentence. Same checks, different distances from the keyboard. The need multiplies when you run several agents in parallel: no human is watching any single turn, so the gate is the only thing holding the line.
What the gate should actually check
A gate is worthless if it checks the wrong thing, and there is a strong pull toward gates that are easy to pass and prove nothing. “It compiles” is a gate an agent clears while shipping the wrong behavior. “The tests pass” is only as strong as the tests, and if the agent wrote the tests in the same breath as the code, the gate is checking whether the code agrees with itself.
The gate should check against something the agent did not author. Acceptance criteria written before generation. A test suite the implementation never saw. A security scan with rules you set. A build that has to produce a running artifact, not just a clean type check. The general principle, the one that separates a real gate from theatre: the check has to be able to fail. If you cannot describe the input that turns it red, it is not verifying anything.
The failure everyone builds first
The common first version of a verification loop is a hook that runs the tests and prints the result, and then the agent reads the result and reports it to you. This is not a loop. It is the same self-reported success with an extra step, because the agent is still the last thing between the check and your decision, and the agent can still summarize a failure as “mostly passing, one minor issue, done.”
The loop only works when the gate acts on the raw verdict directly. The exit code blocks the merge. The failing test output, unedited, is what re-enters the context. The moment a human or an agent gets to paraphrase the verdict before it takes effect, you have reopened the exact gap the loop was built to close.
This is the mechanism underneath the whole idea of verifying AI-generated code with evidence instead of vibes. Not more careful prompts. Not a better model. A loop where success is defined by a check the agent cannot edit, triggered by a hook it cannot skip, enforced by a gate it cannot talk past, repeated until reality agrees.
Where PaellaDoc fits
Wiring this by hand is the part nobody budgets for. You end up being the trigger, running the tests after each session; the gate, deciding whether the failures matter; and the return path, pasting the errors back and asking for a fix. That is you doing the loop’s job manually, one session at a time. PaellaDoc runs the loop for you: checks fire on the events, the gate reads exit codes and not summaries, failures go back to the agent with the real output, and the run does not close as done until the evidence exists. You set what “verified” means once, and stop being the thing standing between a confident sentence and the merge button.
Frequently asked questions
What is a verification loop for coding agents?
It is a four-part mechanism that takes the “did it succeed” decision away from the agent: a trigger fires a check automatically, a check the agent cannot edit produces a verdict from its exit code, a gate acts on that verdict by blocking or passing the work, and a return path feeds real failures back for a fix. It repeats until the check passes or a limit calls in a human. “Tests pass” becomes something the agent has to make true, not something it says.
Why isn’t a hook that just runs the tests and reports the result enough?
Because that leaves the agent as the last thing between the check and your decision, so it can still summarize a failure as “mostly passing, one minor issue, done.” That is self-reported success with an extra step, not a loop. The loop only works when the gate acts on the raw verdict directly: the exit code blocks the merge and the unedited failure output re-enters the context. The moment anyone paraphrases the verdict before it takes effect, the gap reopens.
How do hooks stop an agent from skipping verification?
A hook runs your command automatically on an event, with no cooperation required from the model. Use several layers: a harness hook that fires when the agent finishes a turn or writes a file (tightest, so it self-corrects in-session), a pre-commit or pre-push hook before the change enters history, and integration CI that re-runs on every push independent of anyone’s local setup. Same checks at three distances from the keyboard.
What should a verification gate actually check?
Something the agent did not author. “It compiles” clears while shipping the wrong behavior, and “tests pass” is only as strong as the tests, which are worthless if the agent wrote them alongside the code. Check against acceptance criteria written before generation, a test suite the implementation never saw, a security scan with your rules, or a build that must produce a running artifact. The rule that separates a real gate from theatre: it has to be able to fail. If you cannot describe the input that turns it red, it verifies nothing.