CSE Roadmap

Common tier · about 3 hours · 8 parts

Part 5 of 8 · 2 min read

Planning capacity

Peak load, per-instance limit and room for failure.

Capacity planning turns test results into a number of instances. The steps:

  1. Find the peak load you must serve, including the busiest hour and growth.
  2. Find the safe capacity per instance from the load test, at a level below the knee.
  3. Divide, and add room: for failures (page 5), for releases and for growth.

UDS adds the release point. When a new version rolls out, a fraction of the fleet cannot serve traffic, and the rest must pick up the slack. There must be enough capacity left for the incremental release.

A worked example (made up). Peak is 6,000 requests per second. The test showed an instance reaches its knee at 500, and the team chooses a safe level of 400. Instances needed for load alone: 6,000 ÷ 400 = 15. Room for losing one zone out of three: to survive the loss of a third of the fleet, the remaining two thirds must carry the peak, so the full fleet needs 15 × 3 ÷ 2 = 22.5, rounded up to 23. Room for a rolling release of 10 percent at a time is covered, since 23 − 3 = 20 is still above 15.

Always run the numbers for the case of "one failure unit gone, plus a release in progress, at the peak." (The safe level of 80 percent of the knee is a rule of thumb from common practice, not from the books.)

Autoscaling. UDS ties shedding and levelling to auto-scaling, which adds instances when the service runs hot. Auto-scaling reacts to load, so it takes time. Adding instances during a spike is slower than the spike. It does not remove the need for the protections of page 4, and it does not replace headroom.

Check yourself: 1. In the example, why is the fleet of 23 and not 15? 2. Why must the plan consider a release in progress? 3. Why does auto-scaling not remove the need for load shedding?