CSE Roadmap

Core tier · about 4 hours · 10 parts

Part 9 of 10 · 3 min read

The whole ladder

A single table that sums up the steps, the problem each one solves and the level where it is taught.

The whole ladder

A summary of the ladder, in the order a system usually climbs it. The user counts are loose guides, made up for the example. Real systems climb when measurements show a limit.

Step Added Problem solved Taught in
1 Single server Getting started This page
2 Separate database server Web and data grow separately Level 2
3 Load balancer and several web servers Failure and growth of the web tier Level 1
4 Database replicas Failure and read speed of the data tier Level 4
5 Cache Repeated database work Level 3
6 Content delivery network Slow static files, far-away users Level 1
7 Stateless servers and autoscaling Free choice of server for any request Level 1
8 Several data centres Distance and disaster Levels 1 and 8
9 Message queue Slow work, tight coupling Level 6
10 Logs, metrics, automation Running a large system Level 9
11 Sharding A database too big for one machine Level 4

Two lessons run through the ladder.

  1. Each step has a cost. More parts mean more to run and more that can go wrong. Add a step only when a number shows the need.
  2. Move work to where it is cheapest. Static files to a CDN, repeated reads to a cache, slow work to a queue, heavy data to shards.

A short practice. A site has 20,000 users and one server. It grows to 2,000,000. In which order would you climb? A sensible order: separate database and add a load balancer first (cheap and useful), then replicas and a cache (reads dominate), then a CDN (static files and distance), then a queue (slow tasks), then monitoring (to know what is happening), and last, sharding (only if the data outgrows one leader).

Read it as

When asked "how would this scale?", name the next rung and the number that would trigger it. "At about 5,000 requests per second, the database becomes the bottleneck, so I would add a cache and replicas first."

Check yourself:

  • Which rung would you add if users around the world complain about slow images?
  • Which rung would you add if sending emails makes order pages slow?

Words you will meet

Word Plain meaning
Server A computer or program that answers requests
DNS The phone book that turns names into addresses
Web tier, data tier The request-handling layer and the storage layer
Vertical scaling A more powerful single machine
Horizontal scaling More machines sharing the work
Load balancer Spreads requests across servers
Replication Keeping copies of data on several machines
Leader, follower The copy that takes writes, and the copies that serve reads
Single point of failure A part whose failure stops everything
Cache A fast store of frequent answers
TTL How long a stored copy may be used
CDN Servers around the world that serve static files from nearby
Stateless Servers that remember nothing between requests
Data centre A building of computers
Message queue A safe waiting line for jobs between parts
Sharding Splitting a database across machines
Shard key The rule that picks the shard