Part 9 of 10 · 3 min read
The whole ladder
A single table that sums up the steps, the problem each one solves and the level where it is taught.
The whole ladder
A summary of the ladder, in the order a system usually climbs it. The user counts are loose guides, made up for the example. Real systems climb when measurements show a limit.
| Step | Added | Problem solved | Taught in |
|---|---|---|---|
| 1 | Single server | Getting started | This page |
| 2 | Separate database server | Web and data grow separately | Level 2 |
| 3 | Load balancer and several web servers | Failure and growth of the web tier | Level 1 |
| 4 | Database replicas | Failure and read speed of the data tier | Level 4 |
| 5 | Cache | Repeated database work | Level 3 |
| 6 | Content delivery network | Slow static files, far-away users | Level 1 |
| 7 | Stateless servers and autoscaling | Free choice of server for any request | Level 1 |
| 8 | Several data centres | Distance and disaster | Levels 1 and 8 |
| 9 | Message queue | Slow work, tight coupling | Level 6 |
| 10 | Logs, metrics, automation | Running a large system | Level 9 |
| 11 | Sharding | A database too big for one machine | Level 4 |
Two lessons run through the ladder.
- Each step has a cost. More parts mean more to run and more that can go wrong. Add a step only when a number shows the need.
- Move work to where it is cheapest. Static files to a CDN, repeated reads to a cache, slow work to a queue, heavy data to shards.
A short practice. A site has 20,000 users and one server. It grows to 2,000,000. In which order would you climb? A sensible order: separate database and add a load balancer first (cheap and useful), then replicas and a cache (reads dominate), then a CDN (static files and distance), then a queue (slow tasks), then monitoring (to know what is happening), and last, sharding (only if the data outgrows one leader).
Read it as
When asked "how would this scale?", name the next rung and the number that would trigger it. "At about 5,000 requests per second, the database becomes the bottleneck, so I would add a cache and replicas first."
Check yourself:
- Which rung would you add if users around the world complain about slow images?
- Which rung would you add if sending emails makes order pages slow?
Words you will meet
| Word | Plain meaning |
|---|---|
| Server | A computer or program that answers requests |
| DNS | The phone book that turns names into addresses |
| Web tier, data tier | The request-handling layer and the storage layer |
| Vertical scaling | A more powerful single machine |
| Horizontal scaling | More machines sharing the work |
| Load balancer | Spreads requests across servers |
| Replication | Keeping copies of data on several machines |
| Leader, follower | The copy that takes writes, and the copies that serve reads |
| Single point of failure | A part whose failure stops everything |
| Cache | A fast store of frequent answers |
| TTL | How long a stored copy may be used |
| CDN | Servers around the world that serve static files from nearby |
| Stateless | Servers that remember nothing between requests |
| Data centre | A building of computers |
| Message queue | A safe waiting line for jobs between parts |
| Sharding | Splitting a database across machines |
| Shard key | The rule that picks the shard |