Part 2 of 8 · 1 min read
Capacity and goodput
The most a system can do while still doing it well.
UDS defines capacity as the maximum load the system can withstand. As load grows to that point, performance plateaus or worsens. If load keeps growing, most operations fail or time out. The useful output is the goodput: the part of the requests handled without errors and with low response times. Past capacity, goodput can fall even though the system is busier than ever.
Capacity and goodput
Capacity is the highest load a system can carry while meeting its targets. Goodput is the number of requests per second that are answered correctly and in time.
Example: a service answers 800 requests per second well. At 1,000 it answers only 600 in time, because the rest time out. Its goodput at 1,000 is 600.
Capacity comes from the architecture and from limits such as memory, clock speed, and the bandwidth and latency of network links. Two ways to raise it: scale up (a bigger machine, which hits a wall) and scale out (more machines).
Load is not one number. UDS lists examples: concurrent users, communication links, the ratio of writes to reads. DDIA says to choose the load parameters that suit the system, such as requests per second to a web server, the read to write ratio, or the number of simultaneously active users, and to find which one is the real bottleneck.
Check yourself: 1. What happens to goodput after the system passes its capacity? 2. Name two load parameters other than requests per second. 3. Why does scaling up hit a wall?