CSE Roadmap

Core tier · about 3 hours · 8 parts

Part 6 of 8 · 1 min read

Retry amplification

Retries at every level multiply.

Suppose a client calls service A, which calls B, which calls C. If B's call to C fails and B retries, A waits longer and may hit its own timeout. A then retries, which makes the client wait longer, and the client retries too. UDS calls this retry amplification: with retries at several levels of a chain, the number of calls multiplies, and the deeper the service is in the chain, the more load it sees.

A worked example (made up). Each level allows 3 attempts. One client request can reach C as 3 attempts from the client, each causing up to 3 from A, each causing up to 3 from B, which is 3 × 3 × 3 = 27 calls to C. If only one level retries, C sees at most 3.

UDS's rule for long chains: retry at only one level, and fail fast at all the others. Usually that is the level closest to the user, or the level that knows best whether the call is safe to repeat.

Retry amplification

Retry amplification is the multiplying of calls when several levels of a chain each retry a failing call.

Example: three levels with 3 tries each send up to 27 calls to the last service for one user request.

Check yourself: 1. In a chain of four services where each allows 2 attempts, what is the worst number of calls to the last one? 2. Where does UDS say retries should happen? 3. Why does amplification hurt the deepest service most?