Covers: Results at a 2,000-token budget; The cost of compaction itself; Why a rolling summary loses early facts.
Results at a 2,000-token budget
Five tasks (seeds 0 to 4), 24 probes each; the imagined limit is 16,000 tokens.
Strategy
Probes correct
Mean tokens in window
Summariser tokens (input + output)
Summariser rounds
keep_all
33% (refused on 12 of 24)
10,514
0
0
truncate
1%
1,847
0
0
rolling
42%
1,515
24,349
21.4
clear_tool
17%
1,891
0
0
structured
96%
1,853
0
0
hybrid
93%
1,896
0
0
Probes answered correctly by strategy at the same 2,000-token budget. Keep-all is refused once the request exceeds the imagined 16,000-token limit.
Reading it, in the order an interviewer would probe:
The strategies differ in selection, not size. All five reductions hold the request near 2,000 tokens (rolling runs lower because each fold removes more than it needs to). The 1-to-96 percent spread is entirely what each one chooses to keep.
Truncation fails on exactly the facts that matter. FIFO eviction removes the oldest turns, and the decisions, the constraint and the identifiers were stated early. It scores 0 percent on decisions, constraints, entities and the security decision, and 5 percent on revised decisions (one lucky window).
Clearing tool output alone is not enough here. It is the right first step (it removes the largest, lowest-value content), but once the stubs themselves exceed the budget the FIFO fallback evicts the early user turns. In this task the facts are stated mid-conversation, outside any preserved set, so they are evicted. That is a property of my task, not a refutation of the book.
A rolling summary has the smallest window and the second-best accuracy, at 42 percent, but see section 7 for why, and note it pays 24,349 tokens in summariser calls per task. Its total (window plus summariser) is 175,892 tokens against 185,325 for structured state, so it is slightly cheaper in tokens and less than half as accurate. Output tokens are usually priced higher than input tokens, which I do not model.
Structured state wins because it never re-encodes a fact. A slot is written once and copied verbatim, so retention is independent of the number of steps. The hybrid is a few points lower because stubbing tool output does not help the slots and uses part of the budget on stubs.
The question behind this table
"Your context is full. What do you do?" I'd answer in the order cost goes up. "First I measure tokens against a threshold, and below it I do nothing, because every edit has a cost and can invalidate the prompt cache. Above it I compact: replace already-processed tool outputs and write-tool arguments with stubs that say how much was removed and how to restore it. That is deterministic, free and reversible. If I'm still over, I extract the facts the task depends on (decisions, constraints, identifiers) into a keyed state block that I carry verbatim, and I let the raw history age out. I summarise with a model last, because it costs a call, it's lossy, and a summary that feeds the next summary loses early facts geometrically. I'd measure which of those facts survive with probes before shipping." Then name the trade-off: structured state needs an extractor that can miss things, so I keep a searchable log (module 4) for recall.
The cost of compaction itself
Compaction cost accounting
A strategy's cost is window tokens per step, summed over steps (what you pay to send the request each time) plus the tokens of every summariser call (input read plus output written). A strategy that makes the window small but calls a model to do it has a cost on both sides of the ledger.
The simulation charges each rolling round the tokens of the text it reads and the summary it writes. Over a task that is 24,349 tokens per task, 21.4 rounds, about 1,100 tokens per round. Keep-all pays no compaction cost and about 5.7 times the window tokens of the budgeted strategies (1,051,360 versus about 185,000 per task, summed over the 100 steps). In production the summariser can be a smaller, cheaper model, which shifts the ledger; measure rather than assume.
Why a rolling summary loses early facts
Geometric decay under recursive summarisation
If a summariser keeps a given line with probability p each time it sees it, and the summary is re-summarised k times, the probability that the line is still present is about pk. A 95 percent keep rate sounds excellent and leaves 34 percent after 21 rounds.
The experiment holds the task fixed and varies the summariser's keep rate for salient lines (no drift, to isolate the effect), and separately varies the budget (a larger budget means fewer rounds):
Keep rate p
0.80
0.90
0.95
0.99
1.00
Early facts still in the window at step 100
10%
17.5%
45%
95%
100%
Budget (tokens)
1,500
2,000
3,000
6,000
Summariser rounds
28.4
21.4
14.2
6.4
Early facts retained, rolling summary
15%
22.5%
45%
77.5%
Early facts retained, structured state
100%
100%
100%
100%
Early-fact retention as the per-round keep rate changes. The dashed curve is p^21; later facts see fewer rounds, so the measured line sits above it.
Two honest qualifications. The curve is above p21 because the facts were stated at different steps and the later ones face fewer rounds; the shape is the point, not the exponent. And my "early facts" are the five decisions, the constraint and the two identifiers, scored by presence.
The question behind this experiment
"Why not just summarise as you go?" Because a rolling summary has a recursion in it: each summary is the input to the next, so a fact has to survive every round. At a 95 percent per-round keep rate, a fact summarised twenty times has about a one-in-three chance left. Two mitigations I'd cite: carry the facts that must not be lost outside the recursion, as verbatim structured state, and make the summariser's job smaller by compacting tool output first so it runs fewer rounds. Raising the budget helps for the same reason: fewer rounds.
Identifier drift
A summary is generated text. The simulation lets the summariser change a digit in 2 percent of the salient lines it keeps. In seed 0 it produced B-6867 for the budget code B-6866. A wrong identifier is worse than a missing one, because the agent will use it. The remedy is the same: identifiers go in a slot, copied, never regenerated. Probes that find the wrong value (a stale or forbidden string present) are counted as errors by the judge; a drifted identifier is counted as missing, and the run reports how often the drift hit.