Part 8 of 12 · 1 min read
Answering reliability questions in an interview
Your model provider goes down. What happens?
"Calls fail fast under a timeout, a retry budget stops retries from amplifying load, and the breaker opens so we stop calling the failing provider. The gateway falls back to a second provider or a smaller model, with the user told if the answer is degraded. In the example's simulation that took success from 68 to 99.7 percent during a 120-second outage." Follow-up: "Why not just retry?" Retries alone left success at 70 percent. They also sent 4 calls per request to a dead service.
Explain the circuit breaker.
"Closed: calls flow and I track failures. If the failure rate over the last window crosses the threshold it opens and calls fail immediately for a cooldown. Then half-open: one probe; success closes it, failure reopens it. It stops hammering a failing dependency and gives callers fast failure." Follow-up: "What do you tune?" Window, threshold and cooldown, trading false opens against slow detection.
What is retry amplification?
"Retries multiply across layers: three retries at each of three layers is 64 attempts per request, which can turn a blip into an overload. I retry at one layer, cap retries to roughly a tenth of traffic with a budget, and use backoff with jitter." Follow-up: "Which errors do you retry?" Timeouts, 5xx and 429 with Retry-After. Not 4xx validation errors, and not non-idempotent actions without an idempotency key.
Why bulkheads?
"One hung dependency can hold every shared worker. In the example's test a shared pool let a hung tool drop the healthy tool's success to 3 percent. Separate pools kept it at 100 percent." Follow-up: "Per tool or per tenant?" Both can matter: per tool for dependency failures, per tenant for noisy neighbours (module 7).