Skip to content
Back to all field notes pm · 8 min

Test the riskiest assumption before the agents build the whole thing

When building was slow, you tested the riskiest assumption to avoid wasting months. Now agents build the whole thing before you have identified the assumption at all. That is the new trap.

field note 8 min

The classic discipline is simple. Before you build a feature, find the assumption it depends on that is both most likely to be wrong and most expensive to be wrong about, and test that one cheaply, first. If the whole idea rests on “users will trust us with their bank login” and that turns out false, everything downstream was wasted. So you probe the load-bearing belief before you pour the foundation.

This existed because building was expensive. The riskiest-assumption test was a way to not spend three engineering months discovering, at the end, that the thing nobody could build was the thing everyone assumed. It was insurance against slow, costly construction.

Building is not slow or costly anymore. Which changes the discipline in two directions at once, one that makes it easier and one that makes it more dangerous.

The trap cheap building sets

Here is the failure mode nobody warns you about. When building was slow, the cost of construction was itself a forcing function. You had to think before you built, because building was the expensive part and you could not afford to do it blindly. The friction made you identify the risky assumption, because skipping that step hurt too much.

Now an agent builds the whole feature in an afternoon. The friction is gone, and with it the thing that used to force the thinking. So the new default is to just build it. Why identify the riskiest assumption and design a careful test when you could have the entire feature running by dinner? You skip straight to construction, because construction stopped being the expensive step.

And then you ship a fully-built feature that rests on a belief you never checked. The assumption was still there. It was still the riskiest part. You just never stopped to name it, because building got cheap enough that stopping felt like the slow option. This is exactly how building getting cheap makes deciding get expensive: the cheapness removes the friction that used to do your thinking for you, and now you have to supply that thinking deliberately.

The economics of experiments actually got better

The frustrating part is that cheap building should make riskiest-assumption testing better, not obsolete. The whole reason you tested one assumption instead of ten was that each test cost real engineering. You rationed experiments because experiments were expensive.

That constraint largely lifted. When building a probe is nearly free, you can run the experiment you would never have justified before. The fake door, the throwaway prototype, the two variants shown to ten users, the version that only works for one path so you can watch whether anyone walks it. Things that were too expensive to build just to learn from are now cheap enough to build precisely to learn from and then throw away.

So the economics improved in your favor. You can afford more experiments, faster, on the assumptions that actually matter. The catch is that this only helps if you still do the thinking part, which is naming which assumption is worth an experiment. Cheap building gives you the ability to test more. It does not tell you what to test. That is still your job, and it is the job the cheapness quietly tempts you to skip.

Naming the riskiest assumption still comes first

The skill that did not get automated is identifying, out of everything a feature assumes, the one belief that is most likely wrong and most costly if it is. That is a judgment about your users, your market, and your product, and no amount of build speed produces it.

A useful way to find it: for each thing the feature quietly assumes, ask two questions. How confident am I that this is true? And how much of the whole idea collapses if it is false? The assumption that is both shaky and load-bearing is the one to test first. Usually it is not a technical assumption, because agents make the technical part easy. It is a belief about whether users want this, will trust this, will change their behavior for this, will pay for this. Those are the beliefs a running feature cannot confirm just by existing, and they are exactly the ones cheap building tempts you to assume rather than test.

There is an order to this that the speed keeps scrambling. Identify the assumption, decide what evidence would actually move you, then build the smallest probe that produces that evidence. Cheap building tempts you to invert the order: build first, look at whatever comes back, and reverse-engineer a lesson from it. That inversion launders a build into a finding. You end up treating “we shipped it and nobody complained loudly” as validation of a belief you never stated out loud, which is evidence of nothing except that construction is fast now. The order matters more than ever precisely because nothing external enforces it anymore.

Test the assumption, not the feature

The move is to point cheap building at the assumption instead of at the whole feature. These are not the same thing, and the difference is the entire discipline.

build the feature

  • Whole tail of decisions
  • Conflates many beliefs
  • Hard to throw away

build the probe

  • Isolates one belief
  • Cheap to build
  • Made to be deleted
Aim cheap building at the risky belief, not the whole feature: a probe interrogates one assumption and is built to be thrown away.

Building the whole feature to see if users like it feels like a test, and it is a bad one. It is expensive even at agent speed, because a whole feature carries a whole tail of decisions, states, and polish you now have to maintain. And it conflates a dozen assumptions into one messy signal, so when it does not land you cannot tell which belief was wrong.

Testing the assumption means building the smallest possible thing that interrogates the one risky belief and nothing else. If the risk is “will users trust us with their bank login,” you do not build the whole banking integration. You build the point where they would connect it and watch whether they do. Cheap building makes that isolated probe trivial. The skill is aiming it at the belief, not at the product. Confuse the two and you are back to running a feature factory, shipping fully-built features and calling their existence a learning process.

The other advantage of a probe over a full build is that a probe is cheap to throw away, and throwing it away is the point. A fully-built feature accumulates a gravity nobody planned for. You made it, it works, deleting it now feels like waste, so it stays even when the assumption underneath it turned out shaky. A probe carries no such weight. It exists to answer one question and then die, whichever way the answer came out, which is exactly what keeps the experiment true instead of quietly becoming a commitment.

Where PaellaDoc fits

This is why in PaellaDoc an assumption is something you can name and track, not a belief that stays implicit until it fails. Before the agents build the whole thing, the risky assumptions are explicit, connected to the decision that depends on them and the evidence that would confirm or kill them. Cheap building becomes what it should be, a way to run the experiment that answers the risky question, instead of a way to skip the question and construct on top of an unexamined bet.

The agents will happily build the entire thing before you have thought about which part of it might be wrong. That is the new danger, and it is the mirror image of the old one. You used to protect against wasted construction. Now you protect against unexamined assumptions, because the construction that used to force the examination is free. Name the riskiest belief. Point the cheap build at that, and only that. Then decide.

Frequently asked questions

What is the riskiest assumption test?

Before building a feature, you find the assumption it depends on that is both most likely to be wrong and most expensive to be wrong about, and you test that one cheaply, first. If the whole idea rests on a belief that turns out false, everything built on top of it was wasted. So you probe the load-bearing belief before you pour the foundation.

How do I find the riskiest assumption?

For each thing the feature quietly assumes, ask two questions: how confident am I that this is true, and how much of the idea collapses if it is false. The assumption that is both shaky and load-bearing is the one to test first. Usually it is not technical, since agents make that easy. It is a belief about whether users want it, trust it, or will pay for it.

Should I just build the whole feature to test it now that building is cheap?

No. A whole feature carries a tail of decisions, states, and polish you have to maintain, and it conflates a dozen assumptions into one messy signal, so when it fails you cannot tell which belief was wrong. Aim cheap building at the one risky belief instead: build the smallest probe that interrogates it and nothing else, and is cheap to throw away.

Why does cheap building make this discipline more important, not less?

Because the cost of building used to force the thinking. When construction was expensive, you had to name the risky assumption before you could afford to build. Now an agent builds the whole feature in an afternoon, so the friction is gone and the default is to skip straight to construction, shipping something that rests on a belief you never checked. The experiments got cheaper; naming what to test is still your job.