I asked the Codex app to build an app twice. Neither made it to tomorrow.

Every PaellaDoc demo ends with the same question: “couldn’t Codex build this in ten minutes?”

So I asked the Codex app (the desktop app, not the CLI). Twice: the second time, also asking it to check everything before delivering. All three repos are on GitHub — the first attempt, the second and the PaellaDoc one — so you don’t have to take my word for anything.

The first prompt

This is what I typed into the Codex app on a Saturday afternoon (it’s Spanish, I was in a hurry):

/goal hazme una app de gestión familiar con svelte y fastapi
y sqlite que sea SOTA no me preguntes nada

“Build me a family management app with Svelte, FastAPI and SQLite, make it SOTA, don’t ask me anything.” Model: gpt-5.6-sol. Two more messages after that: “how do I open it?” and “you do it”. Ten minutes total.

PaellaDoc got even less:

Family manager

No spec either way. That’s how people ask for apps.

Ten minutes later

The Codex app delivered “Nido”. The home screen is nicer than PaellaDoc’s, I’ll give it that: warm palette, careful type, a greeting with your name. In a demo video it wins.

Nido’s home screen, exactly as the Codex app delivered it. Screenshot from my July 11 session.

Then I clicked.

Click “Add task”, the app’s primary button. Nothing. The modal never opens.

Click “Tasks”. The menu marks it active and the content stays stuck on the home page.

Check off a task. The write reaches the database, the counter drops… and the screen dies into loading skeletons.

Click back to “Home”. Nothing. The whole app is dead until you hit F5. Reload, try Calendar: same crash.

The menu says Calendar. The breadcrumb and content say Home. The view crashed before rendering.

The console repeats the same thing over and over:

Uncaught Svelte error: invalid_snippet_arguments
A snippet function was passed invalid arguments.
Snippets should only be instantiated via `{@render ...}`
    in Modal.svelte / in App.svelte

And there it stays. Only F5 brings it back.

Four of the five sections. Every create form. The bug is the small part: the project shipped with one check, svelte-check, and it passes with 0 errors on this app. You can have a green build and an app nobody has ever seen start.

What came out the other side

PaellaDoc took three and a half days on the same request. Uglier, yes. But the 59 e2e cases it leaves in the repo drive the app in a real browser: family from scratch, adults and kids with roles, monthly budget per category, expenses with a payer and a responsible member going from pending to paid, kid submits a chore, adult reviews it. There’s a test whose only job is checking that a kid can’t touch anyone else’s tasks.

First run: empty, ready for your family. No seeded fake data. The gate left this screenshot behind while validating the “empty initial dashboard” criterion; I didn’t take it.

Every acceptance criterion leaves its evidence in the repo: a screenshot plus a JSON with what got checked. This one is from today:

{
  "acceptance_criteria": "AC-US-01-4-3-02",
  "verified": {
    "finance": { "budget": "1000.00", "spent": "200.00",
                 "pending": "125.40", "paid": "74.60" },
    "tasks_after_review": { "pending_adult_review": 0 },
    "persisted_after_reload": true
  }
}

1,000 minus 200, 800. 125.40 plus 74.60, 200. To the cent. And the screenshot that sits next to it:

Full page, left by the gate while validating AC-US-01-4-3-02. The numbers match the JSON.

“What if you ask it to verify?” I did.

That’s the objection everyone was going to raise, so I tested it before writing this. Second session, same Saturday, extended prompt:

/goal hazme una app de gestión familiar con svelte y fastapi
y sqlite que sea SOTA no me preguntes nada y comprueba que
todo va perfecto antes de entregármelo

(“…and check that everything works perfectly before delivering it to me.”)

This time the Codex app actually verified. Its session log tells the story: it started the servers, noticed port 8000 was busy and moved the API to 8010, drove the page with Playwright, took screenshots, checked buttons, tested the mobile viewport and fixed a calendar bug it found along the way. Fifteen minutes. It signed off with “implementation and verification completed with no errors or warnings”.

It shows. This time clicking works: I created a task, completed it, deleted it; forms validate; there are loading, error and empty states. It even ships 3 backend tests, passing.

And still. Look at the screenshot slowly:

Taken at 18:45 in the evening: the greeting says “Buenos días” (good morning). The header says Saturday; the week strip puts the 11th under Friday. The mini calendar below, which gets it right, contradicts both.

The app is frozen on the day it was generated:

• “Sábado, 11 de julio” and the “good morning” greeting are hardcoded in the template.
• The “today” panel filters events with a literal startsWith('2026-07-11'). From the 12th on, empty forever.
• The expense form has no date field: the code sends spent_on: '2026-07-11', hardcoded. Every expense you record after July 11 gets booked on July 11. Nobody notices until the month’s total stops adding up.
• The week strip is a hardcoded [7..13] that also puts the 11th under Friday, on a Saturday. Two widgets on the same screen contradict each other, and the visual verification that looked at that screen didn’t see it.

The verification passed because it ran on the same day the app was built. It bought “works today”, literally. And it was one manual pass: it left no script to run it again.

The extras nobody asked for

There’s a clear gradient: the more you talk to the Codex app, the better it gets. Without verification, the app died on the first click; one extra clause, and it survives clicking. More conversation would take it further.

But there’s a ceiling, and it isn’t about effort: generation isn’t deterministic. Every pass adds surface nobody asked for, and that surface looks functional without being wired. Across these two attempts, each verified by hand:

• “Previous / next month” arrows that do nothing.
• A “See full agenda →” link that opens the create-event form, because there is no agenda.
• A decorative event, “Quiet night at home · 20:30”, painted every single day among the real ones.
• A fixed sidebar note, “All calm. The week is well balanced.”, whatever the week looks like.
• A “1,800 € planned” budget (1,200 € in the first attempt) that nobody asked for and can’t be configured.
• A notification bell with a red dot and no notifications behind it.

The PaellaDoc build is the opposite, and that’s the structural difference: the features are the ones that were decided, each ships with its acceptance criterion validated, and there is nothing else. Not one unwired button. Even the empty state on first run is a declared feature, with its own criterion and its own evidence screenshot.

What the tokens buy

I knew to ask for verification, and it wasn’t enough. People who type “build me a family app” don’t even know it’s something you have to ask for. If verification depends on the user knowing to request it, and on a fifteen-minute pass catching everything that can fail tomorrow, there is no verification.

That’s where PaellaDoc’s 582 million tokens go. Writing code took 2.39 million; the rest is the loop running the app, looking at it, and going again, 541 times. The history has commits literally named “Repair e2e validation: adult summary reflects the final state of the full flow”. That commit exists because something didn’t work and the gate wouldn’t let it through. And the gates stay in the repo: the next change takes the same exam. That is the whole point of running an agent inside a gate instead of trusting its own sign-off, which is the comparison I lay out in PaellaDoc vs Codex.

In fairness

Nothing against the Codex app. The speed is real, so is the taste, and the jump from attempt one to attempt two shows it listens: it verified, dodged a busy port, fixed a bug it found. With someone iterating on it, it would get there.

And the experiment is what it is: one app, two prompts, one attempt each, one Saturday. What I measured is the distance between “it checked it for me” and a system where checking is neither optional nor expiring.

Check it yourself

All three repos, published exactly as they came out, code untouched (the only later addition in each is its README):

• gestor-familiar-codex — start the backend and the frontend, click anything.
• gestor-familiar-codex-verificado — open it on any date after July 11, 2026. Check the greeting, check the week strip, add an expense and look at its stored date.
• gestor-familiar-paelladoc — inside: the 59 backend tests, the 59 e2e cases, the 218 commits and the per-criterion evidence.

If you haven’t seen it run, it isn’t done. And having seen it run yesterday doesn’t count.