Part 1 of 7 · 3 min read
Overview
In this module you will
- Define and distinguish compaction (deterministic, reversible replacement of already-processed content by a stub) from summarisation (a model call that rewrites content; lossy and irreversible), and say which one you reach for first.
- Separate storage from presentation: the event log is immutable; the request sent to the model is a projection computed from it.
- Implement six context-reduction strategies behind one interface: keep-all, sliding-window truncation, rolling (recursive) summarisation, tool-output clearing, structured state, and a hybrid.
- Explain, with a measurement, why recursive summarisation loses early facts: retention decays geometrically with the number of rounds, whereas verbatim carry-over is flat.
- Account for the full cost of a strategy: tokens in the window per step, plus the input and output tokens of every summariser call.
- State the prompt-caching trade-off of editing the middle of a prefix.
- Demonstrate summary laundering: a source-agnostic summariser turns an untrusted tool line into trusted text; a source-gated structured state does not.
- Test it: 152 tests, and 33 planted defects to find which mistakes the tests miss.
- Answer "how do you keep a 100-step agent inside its context budget?" the way an interviewer expects: with a policy, a trigger, a cost model and a failure analysis.
Module 1 decided what belongs in the window at one step. A long-horizon agent has a different problem: the same history is re-sent on every step, so the request grows with each tool call until it exceeds the model's limit, and cost and latency grow long before that.
That ordering is sound, but it hides questions an interviewer will ask. What does a summary lose? What happens to a fact that has been summarised twenty times? Does the summariser know where a line came from? What does editing the middle of the history do to the prompt cache? This module answers them with a simulation you can run and break.
About the examples, and what is real
Everything runs offline in code/mini-compact. Real: the six strategies, the budget enforcement, the compaction cost accounting, the trusted-source gate, the judge, the experiments, the 152 tests and the 33 planted defects. All are deterministic.
Imitated:
- The summariser is not a model. It is a function that keeps each line with a fixed probability (92 percent for a decision or a recorded value, 3 percent for anything else) and occasionally changes one digit of an identifier (2 percent). These rates are my assumptions. What the experiments show about them is structural: if a summary is fed into the next summary, retention falls geometrically with the number of rounds, whatever the rates are. The sweep in section 7 shows the effect of changing them.
- The task is a generated 100-step session of about 20,000 tokens, with a handful of facts stated early and a planted instruction in a tool result.
- The structured-state extractor is a short list of patterns standing in for a model.
- Tokens are counted by the rough rule from module 1. The "model limit" of 16,000 tokens is imagined.
- No real model is called, so the numbers describe how designs compare in this setting, not how a real model behaves. The hard exercises ask you to replace the imitation with a real model; I did not run that, so your numbers are the ground truth.