Part 10 of 13 · 2 min read
Mistakes that sink a system-design answer
Common mistakes, and what the example does not do.
Mistakes that sink a system-design answer
- Drawing the final architecture first without targets.
- Adding p95s or using only averages.
- Ignoring the loop's variable length.
- Forgetting evaluation: a router or small model without a quality check.
- Assuming independent failures.
- No cost number.
- Write actions without approval.
- Quoting made-up numbers as facts instead of labelled assumptions.
What the example does not do
About this example What the example does not do
- No measurement: every component number is invented.
- No correlation: outages, rate limits and shared dependencies are ignored.
- No quality model: quality is a fixed number per model. The router's errors and the small planner's losses are not modelled.
- No queueing: concurrency, load and rate limits can dominate real tail latency.
- No streaming: time to first token is not separated from total time.
- Optional experiment: replace each stage's p50 and p95 with values from your own project's traces and see which target fails first.
Check yourself
Your design has ten stages with a p95 listed for each. Can you add the p95s to get the request p95?
Show answer
No. In the simulation the sum of stage p95s was 12.6 s while the simulated request p95 was 10.4 s, because not every stage is slow at once.
For the curious Every box buys something and costs something
Adding a router cut cost from $0.0090 to $0.0052 and p95 from 5.1 s to 4.2 s, but the assumed quality fell from 0.84 to 0.78, below a 0.80 target. The agent loop pushed p95 to 10.4 s against an 8 s target. A smaller planner model got p95 to 7.7 s in the simulation. That says only that it is worth testing, because the model does not capture planning quality. Availability treats failures as independent, so it is an upper bound. A provider outage hits every call at once.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| Clarify the user, job and failure that matters | Before drawing any box. |
| Set targets: quality, p95, cost, availability | Then draw the simplest baseline. |
| Add a box only for a missed target | Name the target each box serves. |
| State your numbers as assumptions | Show how the design changes if they are wrong. |