Part 6 of 12 · 1 min read
How to build it so it holds up
- Give the agent narrow tools: search, read, exact-match edit, run tests; not a general shell by default.
- Protect what defines "correct": tests, CI config, lock files, secrets.
- Sandbox execution with timeouts, output caps and no network unless needed.
- Keep edits small and reviewable; refuse giant diffs and ask the agent to split.
- Hold out tests for evaluation, and also run linters, type checks and the full suite.
- Gate the diff for test edits, skips, deleted assertions and special-cased literals; treat it as a heuristic.
- Bound the loop: an attempt budget, a time budget and a cost budget (module 5 of Level 11).
- Always end with a human-readable pull request; a human approves merges, especially for risky areas.
- Log every tool call with the task and release id (Level 9) so a bad change is traceable (Level 11 module 11).
- Start with low-risk tasks (typos, small bugs, tests, docs) and widen as evidence grows.