CSE Roadmap

Common tier · about 7 hours · 13 parts

Part 2 of 13 · 2 min read

Offload the data to a file and leave only a stub in the window

A tool output does not need to live in every future request just because it lived in this one.

Think of a filing cabinet next to your desk. Instead of keeping every document spread out in front of you, you file it away and keep only a sticky note on your desk saying which drawer it's in.

In plain words: a workspace is a directory the agent reads and writes only through confined file tools. Offloading means saving a large tool output to a file there and leaving only a stub in the window: a short handle with the path, size and a first line. The full data stays retrievable without ever occupying space on every later request.

A worked example. Twelve tool outputs arrive over twelve steps, each about 962 tokens. Offloading each one to a file and replacing it with a stub costs the window about 25 tokens per output instead of 962 -- a 38x reduction before a single question is even asked. Over the whole 12-step, 6-question task, offloading answers 100% of questions for about 2,556 total tokens, against 73,751 for keeping every output inline: a 29x difference, for the same accuracy.

The key lines. From code/mini-workspace/agent.py:

# from code/mini-workspace/agent.py
def stub(o: W.Output, path: str, n_tokens: int) -> str:
    return f"[tool output saved to {path}: {n_tokens} tokens, {o.text.count(chr(10)) + 1} lines; first line: {o.text.split(chr(10))[0]}]"

Read it as: the stub carries exactly four things: the path, a token count, a line count, and the first line. That is enough for the agent to decide whether the file is worth opening. It is deliberately not enough to answer a question from the stub alone; a stub that contained the answer would defeat the entire point of offloading.

Workspace, offloading, stub, handle

A workspace is a directory the agent reads and writes only through confined file tools. Offloading is saving a large tool output to a file and leaving only a stub -- a short text with path, size and first line -- in the window. A handle is anything that lets the agent get the data back later; here, the path.

Example: "saved to out/step02.txt: 943 tokens, 123 lines; first line: # release notes" is a stub doing its entire job in one line -- naming what the file is about, without spending the 943 tokens it's describing.