CSE Roadmap

Core tier · about 4 hours · 12 parts

Part 6 of 12 · 1 min read

What the example found

The example run's timing breakdown, the simulated population, and the sampling comparison.

How to read the example-run table: a single simulated run, one of 20,000, broken down by where its time and cost went.

Item Value
Total duration 3.28 s
Time in model calls (3 spans) 2.94 s
Time in tool calls (2 spans) 0.34 s
Input tokens, summed 5,700
Cost at the assumed price $0.027

What to notice: model calls account for about 90% of this run's total duration, despite being only 3 of its 5 non-root spans. The span-by-name breakdown immediately tells you where to look if this run needed to be made faster. You do not have to read a single line of the agent's own logic.

How to read the sampling table: each row is a different sampling policy applied to the same 20,000 simulated runs, of which 3.9% contained a tool error.

Policy Traces stored Error traces kept Slow traces kept
Head, 1% 1.0% 1% 0%
Head, 10% 9.9% 9% 14%
Tail rule (errors, over 3x median, 1% base) 5.4% 100% 100%

What to notice: head sampling's "error traces kept" column always tracks its "traces stored" column almost exactly, because the decision is blind to outcome. The tail rule stores barely more than half of what 10% head sampling does. Yet it reaches 100% on both error and slow traces. That is the entire case for deciding after the run rather than before it.

{{svg:trace|A root agent span with model-call and tool-call children, each with a duration and token counts.}} {{svg:results|Share of traces stored and of error traces retained under head and tail sampling.}}