Part 4 of 16 · 3 min read
A rolling summary loses early facts geometrically, one round at a time
Summarising a summary is like photocopying a photocopy: each pass loses a little, and after enough passes the original is simply gone.
Think of the telephone game: a message whispered from person to person degrades a little at each hop. After twenty hops, what comes out the other end bears only a passing resemblance to what went in.
In plain words: a rolling (recursive) summary feeds its own previous output back in as input to the next summarisation call, so there is always exactly one summary. This means a fact survives only if it survives every single round. If a summariser keeps a given line with probability p each time, the chance a fact is still present after k rounds is about p to the power of k. That probability falls fast even for a seemingly generous p.
A worked example. At a 95% per-round keep rate -- which sounds excellent -- only 45% of early facts are still in the window after the 21.4 rounds a 2,000-token budget produces over 100 steps; the theoretical p^21 would predict about 34%, and the measured number sits above that only because later facts face fewer rounds than earlier ones. At a 99% keep rate, 95% of early facts survive; at an 80% keep rate, just 10% do. The same structural decay shows up when the budget changes: a 1,500-token budget forces 28.4 rounds and keeps 15% of early facts, while a 6,000-token budget needs only 6.4 rounds and keeps 77.5%.
The key lines. From code/mini-compact/compact.py:
# from code/mini-compact/compact.py
def _roll(self) -> None:
target = max(1, int(self.budget * self.chunk_frac))
take, used = [], 0
while self.buffer and (used < target or not take):
t = self.buffer.pop(0)
take.append(t)
used += tokens.count(render(t))
lines = list(self.summary) + [ln for t in take for ln in render(t).split("\n")]
out = self.summariser(lines, int(self.budget * self.cap_frac))
Read it as: lines = list(self.summary) + ... feeds the previous summary back in as part of the next call's input. This one line is the entire recursion. Every round's output becomes part of the next round's input. That is structurally why the decay is geometric rather than linear, regardless of how good the summariser itself is.
Rolling (recursive) summary, preserved set
A rolling summary is fed into the next summarisation call together with newly aged-out turns, so there is always exactly one summary; a fact survives only if it survives every round. The preserved set is the part of the request never reduced: the system prompt, the original question, and the most recent turns.
Example: keeping identifiers and decisions in a preserved set rather than letting them pass through the rolling recursion at all is the direct fix the module's own interview answer names -- a fact that never enters the recursion cannot decay by p^k, because k never applies to it.