Part 7 of 8 · 2 min read
Where to put the code
One place, not every call.
Where to put the code
UDS advises wrapping each remote call in one library that sets the timeout, applies backoff with jitter, and records metrics. There are libraries in most languages that offer these patterns. The other route is a proxy next to the process that intercepts all outgoing calls. Either way, individual developers do not need to remember to add the protection each time.
The timeout, the retry limit and the backoff cap are linked. A caller's overall timeout must be longer than the sum of its tries and delays, or the outer caller gives up before the inner retries finish. With timeouts of 400 ms, 3 tries and delays of about 1 s and 2 s, the total can reach about 4.2 s. The outer call should therefore allow at least that, or retries should be cut.
Interview angle
Answer in your head first, then open it.
Why set timeouts?
A call without one can hang forever, holding a thread or connection, and can start a cascade.
How do you choose a timeout?
Measure the response time and pick the percentile that matches the false timeout rate you accept.
What is exponential backoff with jitter?
Delays that double up to a cap, with a random part so clients do not retry together.
When should you not retry?
On errors that will not change, and on non-idempotent calls without an idempotency key.
What is retry amplification?
Retries at several levels multiplying the load. Retry at one level only.
Check yourself: 1. Why must the outer timeout exceed the inner retries? 2. Name one benefit of a shared library for remote calls. 3. In the 4.2 second example, how is the total built?
Words you will meet
| Word | Meaning |
|---|---|
| Timeout | The longest wait for a reply |
| False timeout | A call cut off that would have succeeded |
| Backoff | A growing delay between retries |
| Jitter | A random part in the delay |
| Retry storm | Many clients retrying at the same moment |
| Retry queue | A queue holding failed requests to try again later |
| Idempotent | Safe to repeat with the same effect |
| Retry amplification | Multiplied calls from retries at several levels |
| Fail fast | Give up at once and report the error |