CSE Roadmap

Core tier · about 9 hours · 16 parts

Part 13 of 16 · 2 min read

Answering context-compaction questions in an interview

Three model answers, in first person, for the questions this topic reliably draws.

Your context is full. What do you do?

I'd answer in the order cost goes up. First I measure tokens against a threshold, and below it I do nothing, because every edit has a cost and can invalidate the prompt cache. Above it I compact: replace already-processed tool outputs and write-tool arguments with stubs that say how much was removed and how to restore it. That is deterministic, free and reversible. If I'm still over, I extract the facts the task depends on — decisions, constraints, identifiers — into a keyed state block that I carry verbatim, and I let the raw history age out. I summarise with a model last, because it costs a call, it's lossy, and a summary that feeds the next summary loses early facts geometrically. I'd measure which of those facts survive with probes before shipping. Then I'd name the trade-off: structured state needs an extractor that can miss things, so I keep a searchable log for recall.

Why not just summarise as you go?

Because a rolling summary has a recursion in it: each summary is the input to the next, so a fact has to survive every round. At a 95 percent per-round keep rate, a fact summarised twenty times has about a one-in-three chance left. Two mitigations I'd cite: carry the facts that must not be lost outside the recursion, as verbatim structured state, and make the summariser's job smaller by compacting tool output first so it runs fewer rounds. Raising the budget helps for the same reason — fewer rounds.

How can summarisation be an attack vector?

The summariser reads every span with the same trust, so an injected line in a tool result can end up in the summary, which the next request presents as the agent's own notes. By then the provenance is gone. I'd keep provenance on every item, only let trusted roles write the carried state, quarantine and log what was refused, and fence untrusted content in the raw window. I'd also say the gate is necessary and not sufficient: in the example's test it protected the state block but not the two steps before the injected turn aged out.