Part 3 of 13 · 2 min read
A capped, on-demand read answers the question without re-paying for the whole file
Being able to fetch a file back is only a saving if fetching it doesn't mean fetching all of it.
Think of asking a librarian for one specific fact instead of checking out the whole encyclopedia: you get the page you need, and the rest of the volume never leaves the shelf.
In plain words: an on-demand read fetches only the needed part of an offloaded file: matching lines with grep or a line range with read(start, n). There is a hard cap on how many tokens any single result can return. So a model cannot pull the entire file back into the window by accident or by being told to.
A worked example. For the question "What reference code appeared in the release notes?", the agent runs grep('out/step02.txt', 'reference code') and gets back one line: 9: detail: release notes reference code K-2424 -- about 20 tokens, against the 943-token file it was found in. Across all six questions in the worked task, these on-demand reads add about 99 tokens total, a small price for not having carried the full 11,544 tokens of underlying data in the window the whole time.
The key lines. From code/mini-workspace/workspace.py:
# from code/mini-workspace/workspace.py
MAX_READ_TOKENS = 400 # a single read or grep result is capped; the agent must ask for more
def grep(self, name: str, pattern: str, max_hits: int = 5) -> str:
hits = [f"{i}: {ln}" for i, ln in enumerate(self._text(name).split("\n"), 1) if rx.search(ln)]
out = "\n".join(hits[:max_hits])
if len(hits) > max_hits:
out += f"\n[{len(hits) - max_hits} more matches; narrow the pattern]"
return self._cap(out or "[no match]", "narrow the pattern")
Read it as: every result, matched or not, passes through self._cap before it's returned. The cap applies uniformly. So there is no code path where a single grep or read call can return more than MAX_READ_TOKENS, regardless of how many lines actually matched or how the pattern was written.
On-demand read
Fetching only the needed part of an offloaded file with grep (matching lines) or read(start, n) (a line range), with a cap on the result size.
Example: the 400-token cap turning an open-ended "narrow the pattern" result into a bounded one is what keeps on-demand reads cheap even when a pattern accidentally matches far more lines than expected.