CSE Roadmap

Common tier · about 4 hours · 11 parts

Part 6 of 11 · 1 min read

Setting a goal

  1. Write the success criteria as checks. "Report includes three sourced findings" and "tests pass" are checkable; "a good report" is not, unless a rubric and a judge are part of the design.
  2. Separate the claimer from the checker. A different model, a test suite or a person; never the same prompt grading itself.
  3. Set limits per layer: steps per task, tokens, wall-clock time, and number of replans.
  4. Define the escalation path (who, with what context) before you need it, and test it with an impossible task.
  5. Log a progress ledger per step (subgoals done, steps idle, spend), so monitoring is also your trace (Level 9).
  6. Use a cheaper model for monitoring when possible, since it runs on every step; the simulation charged 0.3 model calls per step.
  7. Keep a budget even with a monitor, since monitors fail too.