CSE Roadmap

Core tier · about 9 hours · 16 parts

Part 2 of 16 · 2 min read

A request grows every step until it is refused, and cost rises long before that

The same history gets re-sent on every single step, so a long-running agent's request only ever grows, never shrinks, unless something actively cuts it down.

Think of writing a diary entry at the end of every day by copying out every previous entry first and adding one new line. The amount you write grows every day even though the news itself stays the same size.

In plain words: context budget is the number of request tokens you allow yourself, kept below the model's hard limit because cost and latency both rise before that limit is reached. A projection is the function that computes this step's request from an immutable stored log. The log only ever grows, but the request sent to the model does not have to.

A worked example. A simulated 100-step agent appends an assistant turn and a tool result every step, reaching about 20,000 tokens of log by the end. Sending the whole thing every time (keep_all) crosses an imagined 16,000-token limit at step 79 and gets refused for the remaining 21 steps. Every other strategy in the module holds the request near a fixed 2,000-token budget for all 100 steps, by computing a different, smaller projection of the same underlying log each time.

The key lines. From code/mini-compact/compact.py:

# from code/mini-compact/compact.py
def compact(self) -> None:
    if self.strategy == "keep_all":
        return
    if self.strategy in ("clear_tool", "hybrid") and self.size() > self.budget:
        self._clear_tools()
    if self.strategy == "rolling":
        while self.size() > self.budget and self.buffer:
            self._roll()
    else:
        self._fifo()

Read it as: compact only runs its reduction steps if self.size() > self.budget. The agent does nothing to the request when it is already under budget. This is the threshold-triggered design: reduce only when needed, because every edit has a cost of its own, including to the prompt cache (Part 9 of this module's original material).

Context budget, request, projection

The request is everything sent to the model on one call. The context budget is the number of request tokens you allow yourself, below the model's hard limit. A projection is a function from the stored event log to the request, computed fresh each step; the log is the source of truth.

Example: keep_all being refused at step 79 while every other strategy stays under budget for all 100 steps is the difference between sending the log itself and sending a projection of it -- the log keeps growing in both cases, only the projection is held flat.