CSE Roadmap

Common tier · about 3 hours · 9 parts

Part 6 of 9 · 2 min read

Composition and the price in availability

One call that fans out to many, and why each extra inner call makes the whole less reliable.

Think of a chain of five people, each of whom must pass on a message. If each one fails to pass it on one time in a thousand, the chance that all five succeed is a little lower than for one.

The gateway can compose a reply: it queries several services and merges their answers into one. The book explains why. In a distributed system the data is spread over services, and a use case may need to stitch it back together. Composition relieves the client from knowing which services to ask and cuts the number of requests.

The book also warns that composition is hard to get right, for two reasons.

  1. Availability falls with the number of inner calls, since each call has some chance of failing.
  2. Data may be inconsistent, since some updates have not yet reached all services.

A worked example of the first. Each service is up 99.9 percent of the time (0.999). A call that needs all of them is up only when all are up, so the chances multiply.

Inner calls needed Availability
1 0.999 = 99.9%
3 0.999 × 0.999 × 0.999 = 0.9970 = 99.70%
10 0.999 to the power 10 ≈ 0.9900 = 99.0%

Downtime per 30-day month (43,200 minutes):

  • 99.9 percent allows 0.1 percent: 43.2 minutes.
  • 99.70 percent allows 0.30 percent: 129.6 minutes, about 2 hours.
  • 99.0 percent allows 1 percent: 432 minutes, 7.2 hours.

So a screen that calls ten services, each one "three nines", is down for about 7 hours a month.

Ways to improve:

  • Make some parts optional. If the offers service is down, the page still loads without the offers box. Only the required calls count in the product.
  • Call in parallel. Parallel calls take as long as the slowest one. Calls in turn take the sum. With three calls of 50 ms each, parallel takes about 50 ms and in-turn takes 150 ms.
  • Set time limits. Wait at most, say, 200 ms for each inner call. A slow service must not make the whole screen hang.
  • Fall back. Show a saved copy or a default when an inner call fails (Level 8).

Fan-out

Fan-out is the number of inner calls that one outer call causes.

Example: the home screen call has a fan-out of 3: product, cart and offers.

Read it as

Required calls multiply their failure chances. Optional calls do not.

Check yourself:

  • Five required inner calls are each 99.9 percent available. What is the availability of the outer call, roughly?
  • Why does a parallel call finish faster than a call in turn?