Part 2 of 4 · 7 min read
Definitions
Covers: Definitions; Two views; Results; Design rules; Interview questions, with the follow-ups; Common mistakes; What the example does not do.
Definitions
Outcome eval, trajectory eval
- Outcome eval: scores the final result or the final state of the world.
- Trajectory eval: scores the sequence of steps, such as tool calls and their arguments.
Exact, in-order and any-order match
- Exact match: the steps equal the reference.
- In-order match: the reference steps appear in order, extra steps allowed.
- Any-order match: all reference steps appear, in any order.
Invariant, step budget, milestone
- Invariant: a rule that every acceptable path obeys, such as "verify before refund".
- Step budget: a limit on steps, calls, tokens or cost.
- Milestone: a checkpoint the run should reach; the share reached is a partial-credit score.
Two views
Results
The task: look up the order, verify identity, refund, send email. Invariant: verify before refund. Budget: at most 5 steps. Pass (1) or fail (0):
| Kind of run | Outcome only | Exact path | In order | In order + rules | Rules + budget |
|---|---|---|---|---|---|
| Reference path | 1 | 1 | 1 | 1 | 1 |
| Swapped valid order | 1 | 0 | 0 | 0 | 1 |
| Skipped verification (unsafe) | 1 | 0 | 0 | 0 | 0 |
| Redundant calls (6 steps) | 1 | 0 | 1 | 1 | 0 |
| Wrong refund amount | 0 | 0 | 0 | 0 | 0 |
Diagnostics: milestone score 1.0, 0.75, 0.75, 1.0, 1.0 for the five kinds in the same order; precision, recall and F1 of the tool set were 1.0 everywhere except the skipped-verification run (recall 0.75, F1 0.86). The wrong-amount run scored 1.0 on all path diagnostics.
- Outcome-only has false passes (unsafe luck, waste).
- Exact-path has false failures (valid alternatives).
- Rules plus a budget agree with a human's judgment on all five kinds, but only because I wrote the rules for these five; real tasks need more rules and some will be wrong.
- No path metric replaces the outcome check.
Design rules
- Write the outcome check first, as a state check where you can (the refund exists, with the right amount), not by reading the final text.
- Add invariants for anything unsafe or ordered: must-verify-before, never-delete, never-call-with-these-arguments.
- Add budgets: steps, tool calls, tokens, cost, time.
- Check tool arguments, not only tool names.
- Allow alternatives: use any-order sets for independent steps and several acceptable references for others.
- Keep path similarity and milestones as diagnostics for debugging and ranking runs, not as the gate.
- Record the trajectory in the trace (module 8) so these checks can run on production as well as test data.
Interview questions, with the follow-ups
Why isn't checking the final answer enough for an agent?
"Because the agent acts. It can reach the right end state by an unsafe or wasteful path: in my example a run that skipped identity verification still issued the correct refund, and outcome-only scoring passed it. I add invariants and budgets on the trajectory and keep the outcome as the gate." Follow-up: "Why not compare against a reference trajectory?" Exact matching rejects valid alternatives; my swapped-order run failed it. I specify rules instead of one path.
How would you evaluate tool use?
"Three layers: did it choose the right tool, were the arguments right, and was the sequence acceptable. I check names and arguments against expectations or schemas, enforce ordering and forbidden-call invariants, and count calls against a budget.
What is partial credit for, and what is it not for?
"It helps locate where runs fail and rank near misses, for example a milestone score based on the longest common subsequence with the reference. It should not gate releases: the wrong-amount run got full milestone credit." Follow-up: "Would you ever use an LLM for this?" For fuzzy steps (was the explanation faithful?), calibrated against human labels (next module), not for checks that code can do exactly.
How do invariants relate to safety?
"They turn policies into tests: 'never call refund without verification' is a property of the path, checkable on every test trace and, with the same code, on production traces. It is detection, not prevention; prevention is a gate in the tool layer (Level 10)." Follow-up: "What about unknown unsafe paths?" Run them through monitoring and red-teaming (Level 10) and add each new violation as an invariant.
Common mistakes
- Outcome-only scoring for agents that take actions.
- One exact reference path for tasks with valid alternatives.
- No step or cost budget, so a slow, expensive agent passes.
- Checking tool names but not arguments.
- Using path similarity as the pass criterion.
- Reading the final text instead of the state (the agent said "refund issued" but did not issue it).
- Rules written after the incident, and never added to the suite.
What the example does not do
- No real agent, and each kind of run is fixed, so there are no rates.
- One task and one invariant; real suites have dozens of tools and rules.
- Tool names only: arguments are not modelled.
- No LLM judge for fuzzy trajectory quality (module 4).
- No cost or latency budget, only a step count.
- Optional real experiment (I did not run this): log 50 real runs of an agent, score them with outcome-only and with outcome plus invariants, and count how many outcome passes violate an invariant.
Check yourself
A run skipped identity verification but still issued the right refund. Does an outcome check catch it? (No. It passed the outcome check every time in the five-run example. A rule such as verify before refund catches it.)
For the curious Path scores cannot see a wrong answer
The wrong-amount run got full milestone credit (1.0) and precision, recall and F1 of 1.0, yet the outcome was wrong. Exact-path matching also failed a valid run that swapped two independent steps. So the outcome decides pass or fail, invariants and a step budget guard the path, and milestones are only a debugging aid. The five run kinds were hand-made, so these are illustrations, not measurements.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| Check the end state, not the final text | Does the refund exist with the right amount? |
| Write invariants for ordered or unsafe steps | Verify before refund; never delete. |
| Set a step or cost budget | Two redundant lookups passed every other check; only the budget flagged them. |
| Check tool arguments, not only names | The right tool with the wrong amount is still wrong. |
Draft: confirm what trajectory data your eval tool stores for each run.
What you must be able to say (one page)
- An outcome eval asks "did it end in the right state?"; a trajectory eval asks "how did it get there?" Trajectory evaluation means comparing expected and actual agent steps, and production systems extend judges to tool-call accuracy and reasoning-trajectory validation.
- Outcome-only scoring passes unsafe luck. On my five kinds of run, a run that skipped identity verification but still issued the right refund passed the outcome check 100 percent of the time. Only a rule ("verify before refund") caught it.
- Exact-path matching is too strict. A valid run that swapped two independent steps failed exact-path and even in-order matching; the rule-based scorer (invariants plus a step budget) accepted it. Over-specified trajectories punish correct alternatives, so specify rules, not one path.
- Rules need a budget too. A run with two redundant lookups (6 steps instead of 4) passed outcome and in-order checks; only the step budget (at most 5) flagged it.
- Path metrics cannot see wrong answers. The wrong-amount run scored full milestone credit (1.0, all steps in order) and precision, recall and F1 of 1.0, yet the outcome was wrong. Always score the outcome as well.
- Partial credit helps debugging, not gating. Milestone score (the share of reference steps reached in order, via the longest common subsequence): 1.0 for the reference run, 0.75 for the skipped-verification run.
- Recommended structure: the outcome decides pass or fail; invariants (must-happen-before, never-call), a step or cost budget and tool-argument checks guard the path; milestones and path-similarity metrics are diagnostics.
- Honest limits: the five run kinds are hand-made, so the pass rates are deterministic illustrations, not measurements of any agent.