CSE Roadmap

Core tier · about 3 hours · 12 parts

Part 10 of 12 · 1 min read

Quick notes

  • An outcome eval asks "did it end in the right state?"; a trajectory eval asks "how did it get there?"
  • Outcome-only scoring passes unsafe luck: a run that skipped identity verification but issued the right refund passed every outcome check and only a rule caught it.
  • Exact-path matching is too strict, because it punishes correct alternatives such as two independent steps swapped. Specify rules, not one path, and add a step or cost budget to catch redundant work.
  • Path metrics cannot see wrong answers, so always score the outcome as well. Partial credit helps debugging, not gating.
  • Recommended structure: the outcome decides pass or fail; invariants, a budget and tool-argument checks guard the path; milestones and path-similarity metrics are diagnostics.