Skip to content
Back to all field notes verify · 8 min

Reviewing code you didn't write: scope first, diff last

Opening the diff first is how you review AI code badly. You end up checking whether the code is well written instead of whether it should exist. Start with scope.

field note 8 min

You open the pull request an agent produced. Four hundred lines across nine files. You do what you have always done with a pull request: you start reading the diff, top to bottom, judging each hunk. Forty minutes later you have read most of it, corrected some naming, and you approve it, with the vague awareness that you never actually confirmed it does the right thing. You checked that the code was written competently. You never checked whether it should exist in that shape at all.

That habit, diff-first, is fine for a colleague’s change and wrong for an agent’s. Reviewing code you didn’t write, at the volume agents produce it, needs the opposite order. Scope first. Diff last. Sometimes never.

Why the colleague habit fails here

When a teammate sends you a pull request, you bring a pile of assumptions that make diff-first reading work. You assume they understood the ticket, because they were in the standup. You assume they respected the architecture, because they have been burned by it before. You assume the change is scoped to the thing you discussed, because a person who touches an unrelated module usually mentions why. Those assumptions let you skip straight to the lines, because the expensive questions, intent and scope, are already answered by shared context.

None of that context exists with an agent. It was not in the standup. It has no scar tissue. It will cheerfully refactor a file you never mentioned because it seemed tidy, and it will not tell you unless you look. So the questions you got to skip with a human, “is this even the right change” and “what did it touch that I didn’t ask about,” are wide open, and they are precisely the questions the diff answers last and worst. The diff shows you the code that changed. It hides everything the change was supposed to leave alone.

Scope first: is this the right change at all

Before a single line, answer one question. What was this supposed to do, and does the shape of the change match that? Not the implementation, the shape. Which files should a correct version of this touch? Roughly how big should it be? Does the diff live where you expected, or has it sprawled into places that have nothing to do with the task?

This is a thirty-second check and it catches the most expensive errors, the ones no amount of line-reading finds, because they are errors of the whole. An agent asked to add a feature that instead rewrites an existing one. A change that is three times the size the task warranted, which almost always means it did something you didn’t ask for. A diff that touches the auth module when the task was about export formatting. You cannot see “this solved the wrong problem” by reading the solution carefully. The solution can be immaculate. You see it by holding the change up against the intent before you get seduced by the code.

If scope is wrong, stop. Do not review the diff. Send it back. Reviewing the line-by-line quality of a change that should not exist is the purest waste of attention there is, and it is where diff-first reviewers lose most of their day. The cheaper version of this review happens before the code exists, on the spec itself.

Blast radius: what did it touch that you didn’t ask about

Scope passed. Now, before the lines, the second widest question. What is the blast radius? What existing behavior could this change have altered, whether or not the diff makes that obvious?

This is the region where agent-written code actually hurts, and it is nearly invisible in the diff. A model writes each piece to be locally correct and has no durable sense of the invariants holding the rest of the system together. So it changes a shared function to suit the new caller and silently shifts the behavior of five old ones. The diff shows you a clean edit to one function. It does not show you the five callers that depended on the old behavior. That is the locally correct, globally incoherent failure, and reading the changed lines harder will never surface it, because the damage is in the lines that did not change.

The blast radius is not read, it is tested. The integration tests over the touched area, the type checker across the callers, the suite that exercises the modules downstream of the edit. This is the part of review that does not scale by human eyes at all, and trying to do it by eye is how large agent changes slip regressions past careful reviewers. The reviewer’s job here is not to trace every caller by hand. It is to confirm that something mechanical did, and to look where the machine says it changed something it shouldn’t have.

Diff last: and only the parts that carry risk

Now, and only now, the lines. And even here, not all of them equally. With human review you read the whole diff partly to keep shared understanding alive across the team. With agent code that social function is gone, and reading all four hundred lines with equal attention is not diligence, it is a way to exhaust yourself into missing the ten that matter.

Spend the attention where a defect is both likely and costly. The boundary code, where the change meets the outside world or the rest of the system. The security-relevant surface, input handling, auth, anything touching data that leaves the machine, the kind of thing the OWASP Top Ten exists to remind you an agent will get subtly wrong while looking completely reasonable. The parts the specification actually cared about. Skim the boilerplate the agent generated, because it is boilerplate and the tests cover it, and pour your reading into the handful of places where being locally plausible and being correct come apart.

The order is the whole technique

Widest to narrowest. Is this the right change, then what else did it touch, then are these specific lines correct. Each stage can stop the review, and the early stages are cheap and catch the expensive things. Diff-first inverts this. It spends your most expensive, least scalable resource, careful line reading, first and evenly, on a change you have not yet confirmed is the right change touching the right things. At agent volume that inversion is fatal. The code arrives faster than you can read it, and if reading it is your first move, you fall behind on day one and start approving things you skimmed.

Reviewing code you didn’t write is one of the core motions of verifying AI-generated code, and the discipline is entirely in the order. Cheap wide checks up front, expensive narrow reading at the end, and a willingness to reject at any stage without descending into the lines. That is how you review more code than you could ever read, and still know what you approved.

Where PaellaDoc fits

Scope-first review needs something to check the scope against. If the only record of what the change was supposed to do is a one-line ticket and a chat that scrolled away, “is this the right change” is a question you cannot actually answer, so you default to reading the diff. PaellaDoc keeps the intent, the acceptance criteria, and the affected surface as artifacts next to the code, so the widest questions have a fixed reference, and the blast radius is checked by tests that run on the change instead of traced by eye. Your review starts where it should, at scope, and your reading time lands on the few lines that carry real risk instead of the four hundred that don’t.

Frequently asked questions

How do I review code an agent wrote?

In the opposite order to a colleague’s pull request: widest question first. Confirm scope, whether this is even the right change in the right places. Then blast radius, what existing behavior it could have altered, checked by tests rather than by eye. Then, and only then, read the lines, and only the ones that carry risk. Each stage can stop the review, and the cheap early stages catch the expensive errors line-reading never finds.

Why doesn’t reviewing AI code work like reviewing a colleague’s PR?

Because the assumptions that make diff-first reading work are gone. A teammate was in the standup, respects the architecture, and mentions why they touched an unrelated module, so intent and scope are answered by shared context. An agent has none of that. It will refactor a file you never mentioned and not tell you. The questions you skip with a human, “is this the right change” and “what did it touch,” are wide open, and the diff answers them last and worst.

What should I check before reading the diff?

Scope, in about thirty seconds. What was this supposed to do, and does the shape of the change match? Which files should a correct version touch, roughly how big should it be, has it sprawled into unrelated places? This catches errors of the whole, an agent that rewrote an existing feature instead of adding one, a diff touching auth when the task was export formatting. If scope is wrong, stop and send it back. Reviewing the quality of a change that shouldn’t exist is pure wasted attention.

Do I need to read every line of an agent’s pull request?

No. Reading all four hundred lines with equal attention is not diligence, it is how you exhaust yourself into missing the ten that matter. Spend attention where a defect is both likely and costly: boundary code, the security-relevant surface, input handling, auth, anything touching data that leaves the machine, and the parts the specification actually cared about. Skim the boilerplate the tests already cover, and pour your reading into where locally plausible and actually correct come apart.