Part 11 of 13 · 1 min read
Quick notes
- Use a fixed order: clarify the user, job and failure that matters; set targets; draw the simplest baseline; add boxes only when a target is missed; walk through failure modes and approvals; say how you would evaluate and monitor; state the trade-offs.
- Percentiles do not add. Summing stage p95s overstates the tail, while retry timeouts push the real tail up. Serial availability multiplies, and retries raise it. A provider outage breaks the independence assumption.
- Find the bottleneck by perturbation: halve each stage's latency in turn and see which one moves the p95. Optimise what the experiment shows, not what feels slow.
- Every box buys something and costs something, and fixes are hypotheses: a smaller planner model may fix latency and cost while losing planning quality the simulation cannot see.
- Defend the numbers as assumptions and show how the design changes if they are wrong. Stream progress, because time to first token is long when an agent plans and calls tools first.