Part 7 of 13 · 2 min read
Answering files-and-scratchpads questions in an interview
How do you stop tool output from filling the context window?
"I don't keep it in the window. I write the full output to a file in a sandboxed workspace and leave a stub with the path, the size and the first line. If the agent needs a detail it greps or reads a line range, with a cap on the result. In the example's test, twelve outputs of about a thousand tokens answered everything correctly for about 2,600 tokens, against about 74,000 for keeping them all, and the cost stayed flat when the outputs got four times longer." Follow-up: "Isn't an extra read a cost?" Yes: 99 tokens over six questions here. It is paid only when asked, and it is small compared with carrying every output on every step. Follow-up: "What if the model doesn't know to look?" Then the stub needs to say what the file contains; a title is the minimum, and a short plan or notes entry helps.
Scratchpad or file store: which, and when?
"A scratchpad is for what the agent judged important when it wrote it: decisions, findings, task status. It's always in the window, so it has to stay small, and it can't answer questions about anything unflagged. A file store holds the raw data and answers anything, at the cost of a read. In the example's test notes alone scored 50 percent, answering every flagged question and no unflagged one, and files scored 100. I'd offload by default and add notes only for facts that are needed repeatedly." Follow-up: "Who decides what to flag?" The model, at write time, with the errors that implies; that is why the raw data stays on disk.
What are the security risks of giving an agent file tools?
"Path traversal, symlinks, unbounded reads, partial writes, and hostile contents. I resolve every path and require it to be under the root, which also catches symlinks; cap every read; write to a temp file and rename; and treat anything read back as untrusted text that gets fenced in the window and cannot write memory. Beyond that I'd run the workspace in a sandbox with quotas and no network, because the path check protects the file tools, not the machine."