Core tier · about 6 hours · 13 parts
Part 7 of 13 · 1 min read
The answer, step by step
Adding boxes one at a time: p95, cost and assumed quality.
- Clarify. Who is the user? What is the job? What is the worst failure (wrong refund, leaked data)? What volume? Is a write action involved?
- Targets. Put numbers on quality, p95, cost and availability. If the interviewer gives none, propose them and say so.
- Baseline. One model call plus retrieval if the answer needs data. Estimate its latency, cost and quality.
- Add boxes for a reason. Guardrails for risk, router for cost, cache for repeated queries, a loop only when one pass cannot do the job.
- Failure modes. Timeouts, tool errors, bad plans, loops that never end, prompt injection, write actions that need approval.
- Evaluate and monitor. Offline set, online sampling, traces, the three observability metrics above.
- Trade-offs. Say what you gave up and what would make you reverse a choice.