Part 3 of 8 · 1 min read
Retries and backoff
Try again, but more slowly each time.
When a call fails or times out, the client has two choices: fail fast, or try again later. If the cause was a short blip in connectivity, a retry after a pause is likely to work. If the server is overwhelmed, an immediate retry makes things worse. So retries must be slowed down, with longer and longer delays, until a maximum number of tries or a maximum total time is reached.
UDS describes a capped exponential backoff:
delay = min(cap, initial_backoff × 2^attempt)
With a cap of 8 seconds and an initial backoff of 2 seconds, the book's delays are 2 seconds, then 4, then 8, and then 8 for every later try, because the cap holds them.
Exponential backoff
Exponential backoff means that the delay between retries doubles after each failure, up to a maximum called the cap.
Example: retries wait 2, 4, 8, 8, 8 seconds when the cap is 8.
Retries do not have to be live waits. In batch jobs with no strict real-time need, a failed request can be parked in a retry queue (Level 6) and read again later by the same process or another.
Check yourself: 1. With an initial backoff of 1 second and a cap of 20, give the first six delays. 2. Why must the delay grow? 3. When is a retry queue a better choice than waiting?