CSE Roadmap

Core tier · about 6 hours · 13 parts

Part 11 of 13 · 1 min read

Quick notes

  • Answer the hardest-bug question in six beats: symptom, reproduction, localisation, hypotheses tested, cause and fix, and what you changed so it cannot recur.
  • Agent failures are quiet. A tool succeeds but misleads, an answer sounds fluent but is wrong, a plan half works. Everything starts with a trace with spans, tool arguments and statuses.
  • Localise before you theorise: compare the failing trace with a healthy run of the same task and find the first span that differs. A fluent wrong answer cannot be localised this way; it needs an evaluation.
  • Rules catch structural faults, not quality faults, and precision is where they hurt. Read your false positives; they show which two causes share a symptom. A threshold is a trade-off between missed faults and wasted reviews.
  • Change one thing, confirm on a held-out set, and turn the failure into a regression test. Read twenty failing traces before building tooling.