Compaction eats your decisions: managing the context window on long work

Here is a failure I have watched more than once. Early in a long session, I tell the agent a constraint: this service has to stay idempotent, the provider retries. Three hours later, deep into the work, the agent writes a handler that is not idempotent. It is not being careless. It genuinely does not know the constraint anymore, because the window filled up, the session compacted, and my instruction from hour one got summarized into oblivion. The decision that made the whole task correct is gone, and nothing announced its departure.

That is the quiet cost of the context window. It is a fixed-size container, and a long run produces more material than fits. Something has to give, and what gives is usually the thing you said once, early, and never repeated: the constraint, the decision, the reason. The agent does not stop when it forgets. It keeps going with the same confidence, now optimizing against a contract it can no longer see. On any job long enough to matter, the window stops being infinite and starts being a budget you have to manage.

What compaction actually throws away

When a session runs out of window, the tool compacts: it summarizes older turns to make room. Useful, and necessary, but summarization is lossy by design, and it is not lossy at random. It keeps what looks locally important in the recent flow and drops what looks like old detail. A one-line constraint you stated three hours ago and never mentioned again looks exactly like droppable old detail, even when it is the single most load-bearing sentence in the whole run.

So compaction has a bias: it preserves the shape of the recent conversation and erodes the decisions that anchored the start. The early “we must never do X” gets compressed to nothing while the last twenty minutes of implementation detail stays crisp. This is the mechanism behind a whole class of long-run failures, and it is why a run drifts in the middle rather than crashing: the agent did not break, it forgot, and then kept building confidently on the gap.

Anthropic’s own account of effective context engineering for AI agents makes the underlying point plainly: attention is finite, and more tokens in the window is not the same as more understanding. Past a point, adding context degrades the agent’s grip on what matters instead of improving it. The window is a resource with a ceiling, and treating it as unlimited is how you get confident amnesia.

The window is a budget, not a warehouse

The mental shift that fixes most of this is to stop treating the context window as storage and start treating it as a budget you spend on purpose. A warehouse is where you put everything and assume it will be there later. A budget is finite, and every token you spend on noise is a token not available for the decision that matters.

Two things follow from that framing.

First, what goes into the window is a choice, not a default. Dumping the whole repo, the full history, and every rules file into every call is not thoroughness, it is spending your budget on things the agent does not need for this task, which is the core argument of context engineering for coding agents: curate what the agent sees, do not flood it. A precise slice of the right files beats the entire tree.

Second, the decisions that must survive cannot live only in the window. If the only copy of “this must stay idempotent” is a message in the transcript, compaction can and will eat it. The fix is not to repeat it louder. It is to keep it somewhere the window does not own, a file, a task, a contract, and re-supply it when it matters, so the constraint outlives any single summarization. This is the in-session twin of context loss between sessions: the same failure on a shorter timescale, with the same fix.

Budgeting the window on long work

Concretely, here is how I keep long runs from forgetting themselves, all of it doable today:

• Keep the contract outside the transcript. The constraints and acceptance criteria live in an artifact, not just in something the agent said early. That way compaction cannot delete them, and any resumed or handed-off session can re-read them. This is the same reason a clean handoff carries a distilled bundle instead of the raw history.
• Spend the budget on the task, not the archive. Load the files this step needs, not the whole repo. A smaller, sharper context both fits better and produces better output.
• Re-anchor at boundaries. At each episode boundary, restate the live constraints into the working context. It costs a few tokens and buys you an agent that still knows the rules.
• Watch the rules files. Every line in a rules file the agent auto-loads is budget spent on every call. A bloated, half-stale rules file is not just misleading, it is expensive, crowding out the context the task actually needs.
• Checkpoint before compaction bites. If a run is approaching the ceiling, a checkpoint with the current state written down means a fresh session can continue from a clean, small context instead of a compacted, lossy one.

The pattern under all of it: the window holds the working set, not the source of truth. Anything that has to survive the whole run lives outside the window and gets re-supplied on demand.

Where PaellaDoc fits

Compaction eats decisions because the window is finite and summarization is biased against the old constraint that anchored the work, and the fix is to keep what must survive outside the window and re-supply it when it counts. That is the mechanism, and it is why I do not think a bigger window alone solves this: a larger budget still fills, and still erodes the anchor first, which is the same reason agents need a runtime, not a bigger model. PaellaDoc keeps the contract and the decisions as artifacts outside any session’s window and feeds the agent the slice it needs, so a long run is held against constraints it cannot forget rather than ones it summarized away. The person re-anchoring the agent by hand every time it drifts is the runtime, until the window is managed for them.

FAQ

Why does my AI agent forget instructions during a long session?

Because the context window is a fixed size, and when it fills the session compacts by summarizing older turns. Summarization is lossy and biased toward keeping recent detail, so an early one-line constraint you never repeated is exactly the kind of thing that gets erased, while recent implementation chatter survives. The agent then keeps working without the instruction, unaware it is gone.

Does a bigger context window fix this?

It raises the ceiling but does not change the mechanism. A larger window still fills on long work, and compaction still erodes the early anchoring decisions first. More tokens also is not the same as more understanding, past a point extra context degrades the agent’s focus. The durable fix is to keep must-survive constraints outside the window and re-supply them, not to rely on a bigger warehouse.

How do I stop compaction from losing important decisions?

Keep them out of the transcript. Store constraints and acceptance criteria in an artifact the window does not own, re-state the live ones at each episode boundary, checkpoint before you hit the ceiling, and load only the files a step actually needs instead of flooding the window. The window should hold the working set; the source of truth lives outside it.