CSE Roadmap

Common tier · about 5 hours · 12 parts

Part 6 of 12 · 1 min read

How to build it so it holds up

  • Give the agent narrow tools: search, read, exact-match edit, run tests; not a general shell by default.
  • Protect what defines "correct": tests, CI config, lock files, secrets.
  • Sandbox execution with timeouts, output caps and no network unless needed.
  • Keep edits small and reviewable; refuse giant diffs and ask the agent to split.
  • Hold out tests for evaluation, and also run linters, type checks and the full suite.
  • Gate the diff for test edits, skips, deleted assertions and special-cased literals; treat it as a heuristic.
  • Bound the loop: an attempt budget, a time budget and a cost budget (module 5 of Level 11).
  • Always end with a human-readable pull request; a human approves merges, especially for risky areas.
  • Log every tool call with the task and release id (Level 9) so a bad change is traceable (Level 11 module 11).
  • Start with low-risk tasks (typos, small bugs, tests, docs) and widen as evidence grows.