Someone always asks the same question once they have a local model running: why keep paying for the cloud one? The honest answer is not about money, and it is not about speed. It is about what the model gets wrong, and that part is invisible until the task is big enough to expose it.
I run both, on purpose, and the question I actually answer every time is not “which model is better.” It is “what am I about to ask it to do, and does this task sit in the zone where the difference stops mattering.”
This is not the cloud-vs-local-tools question
Cloud vs local AI dev tools is about where your product’s state lives: your specs, your decisions, your product graph, on your disk or on someone’s server. That trade is real, and it has nothing to do with which model writes your code. A tool can be local-first and still call a frontier model for inference, because the coordinating system and the engine behind a single call are different things.
This is the other question, the one that stays even after you have settled where your data lives: if the engine itself is a small model running on your machine instead of a frontier model in the cloud, what do you actually lose in the code it produces.
What the gap is not
It is not latency. PAELLADOC’s own inference benchmark module measures first-token time, generation speed and failures for the routes you actually use, and it says so plainly: a latency run does not know whether the code is correct, whether a plan is good, or whether the model handled a tool safely. A local model can answer fast and still answer wrong. A slow frontier model can be the one you should have trusted.
It is not price either, not directly. Controlling AI costs is about not sending a copy edit to your most expensive engine. That heuristic assumes the cheap route is adequate for the easy task, which is usually true. It says nothing about what happens once the task stops being easy.
Where the gap actually lives
Three places, in my experience, and none of them show up in a short demo.
Multi-step reasoning under ambiguity. A well-specified task with a test already written is closer to transcription than to judgment, and a smaller model does transcription fine. The gap opens when the spec is incomplete and the model has to decide what you probably meant. A frontier model tends to ask, or to make a defensible guess and flag it. A smaller local model is more likely to commit confidently to the first interpretation, which is the exact shape of false success: a plausible result that is quietly wrong.
Agentic tool use. Reading an error message, deciding whether to retry, modify the approach, or stop and ask, across ten or fifteen steps without losing the thread, is a different skill from writing a function. This is where agents will drift from a plan fastest with a weaker engine underneath, because recovering from a bad step takes the same judgment that caused the step in the first place.
Cross-file context at scale. A change that touches one function is local work. A change where the right answer depends on an invariant three files away is a different task, and it is exactly the kind of task SWE-bench was built to test honestly, with real GitHub issues and pull requests pulled from real repositories instead of a toy problem written for the benchmark. That is the right instinct to borrow even if you never run the public benchmark yourself: judge by whether the task got done against a real codebase, not by how confident the output sounds.
Where the gap barely matters
Flip it around, because the gap is not a tax on every task, only some of them.
A narrow edit with a test already pinning the expected behavior. Formatting, boilerplate, a CRUD endpoint that follows a pattern already established ten times in the repo. Anything where acceptance criteria already exist and the model’s job is to satisfy them rather than invent them. On that kind of task, a local model finishing in a few seconds, for free, is not a compromise. It is the right tool, and paying a frontier model to do transcription is the waste, not the other way round.
And there is a case where the gap is irrelevant for a different reason entirely: when the work cannot leave your machine at all, for a client, a contract, or a jurisdiction. Running local for privacy is a requirement, not a trade-off, and on that axis the quality gap is a price you pay on purpose, with your eyes open, for a guarantee no cloud model can give you.
How you actually find out, for your own repo
Nobody’s benchmark answers this for you, because the gap depends on your codebase’s shape, your spec quality, and the specific tasks you throw at it. The honest version is unglamorous: run the task that matters against both routes, on your own repo, and check the outcome against a gate, not against how fluent the explanation sounds.
That is the shape of the decision every time, not a one-time verdict about which model wins. Model routing is the mechanism for acting on it without re-deciding by hand on every task: route the narrow, well-specified work to the cheap and local engine, and reserve the frontier call for the task where ambiguity or a long agentic loop makes the gap real money. It is a small, constant version of being the runtime: not writing the code, but deciding, task by task, which engine has actually earned this one.
What closing the gap actually costs
You can narrow the gap with more scaffolding: tighter prompts, more examples, a longer system message, a retry loop that catches the model’s own mistakes. All of that is real engineering time, and it buys back only part of the difference, because a chunk of the gap is reasoning depth, not instruction-following, and no amount of prompt tuning turns a smaller model into a bigger one.
The question worth asking is not “can I close the gap” but “is this specific task cheap enough, and the stakes low enough, that I would rather spend five minutes engineering a workaround than send it to the model that would have just done it.” Past a certain ambiguity or a certain blast radius, the answer is no, and that is fine. The point of routing is never having to re-litigate it.
Where PaellaDoc fits
PaellaDoc treats this as a routing decision, not a one-time commitment to a model. The fleet includes local engines through Ollama and frontier models from any vendor, and the router picks the engine for the task in front of it rather than betting the whole project on one answer to “which model is better.” The inference benchmark module tells you what your own configured routes actually do before you lean on either side of the trade.
Where do you draw your own line between local and frontier? Tell me on the forum.
Frequently asked questions
Is a local LLM as good as a cloud LLM for coding?
It depends on the task, not on a verdict about the models in general. For narrow, well-specified work with a test or acceptance criteria already pinning the expected result, a local model usually finishes the job fine. The gap opens on ambiguous specs, long agentic tool-use loops, and changes whose correctness depends on something several files away. Those are the tasks where a weaker model is more likely to commit confidently to a wrong answer instead of flagging the ambiguity.
What do I actually lose by running a local model instead of a frontier one?
Not speed, and not directly money. You lose depth of reasoning under ambiguity, reliability across a multi-step agentic loop where the model has to interpret its own errors and decide whether to retry or stop, and robustness on changes that depend on context several files away from the one being edited. On narrow, test-pinned tasks, you lose nothing that matters.
How do I know if the gap matters for my specific project?
Run the task that actually matters to you through both routes and judge the outcome against a real gate — tests passing, acceptance criteria met — not against how confident or fluent the explanation sounds. A local benchmark of your own configured routes, measuring real outcomes on your own repo, tells you more than any public leaderboard, because the gap depends on your codebase’s shape and your spec quality, not on a model’s general reputation.
Can I close the gap with better prompting?
Partly. Tighter prompts, more examples and a retry loop that catches obvious mistakes buy back some of the difference on instruction-following. They do not buy back reasoning depth, because that part of the gap is not about how the task is phrased. Past a certain level of ambiguity or stakes, routing the task to a stronger model costs less than engineering around a smaller one’s limits.