CSE Roadmap

Core tier · about 6 hours · 13 parts

Part 10 of 13 · 2 min read

Mistakes that sink a system-design answer

Common mistakes, and what the example does not do.

Mistakes that sink a system-design answer

  • Drawing the final architecture first without targets.
  • Adding p95s or using only averages.
  • Ignoring the loop's variable length.
  • Forgetting evaluation: a router or small model without a quality check.
  • Assuming independent failures.
  • No cost number.
  • Write actions without approval.
  • Quoting made-up numbers as facts instead of labelled assumptions.

What the example does not do

About this example What the example does not do
  • No measurement: every component number is invented.
  • No correlation: outages, rate limits and shared dependencies are ignored.
  • No quality model: quality is a fixed number per model. The router's errors and the small planner's losses are not modelled.
  • No queueing: concurrency, load and rate limits can dominate real tail latency.
  • No streaming: time to first token is not separated from total time.
  • Optional experiment: replace each stage's p50 and p95 with values from your own project's traces and see which target fails first.

Check yourself

Your design has ten stages with a p95 listed for each. Can you add the p95s to get the request p95?

Show answer

No. In the simulation the sum of stage p95s was 12.6 s while the simulated request p95 was 10.4 s, because not every stage is slow at once.

For the curious Every box buys something and costs something

Adding a router cut cost from $0.0090 to $0.0052 and p95 from 5.1 s to 4.2 s, but the assumed quality fell from 0.84 to 0.78, below a 0.80 target. The agent loop pushed p95 to 10.4 s against an 8 s target. A smaller planner model got p95 to 7.7 s in the simulation. That says only that it is worth testing, because the model does not capture planning quality. Availability treats failures as independent, so it is an upper bound. A provider outage hits every call at once.

What you will use daily at work

Practice What it looks like
Clarify the user, job and failure that matters Before drawing any box.
Set targets: quality, p95, cost, availability Then draw the simplest baseline.
Add a box only for a missed target Name the target each box serves.
State your numbers as assumptions Show how the design changes if they are wrong.