CSE Roadmap

Core tier · about 4 hours · 10 parts

Part 4 of 10 · 2 min read

Scale up or scale out

The two ways to add power, and the load balancer that makes the second way work.

Think of a delivery van. The business can buy one bigger van, or add more vans.

Vertical scaling

Vertical scaling, or scaling up, means adding more power to one machine: more processor, more memory, more disk.

Example: replacing a server that has 16 GB of memory with one that has 256 GB.

Horizontal scaling

Horizontal scaling, or scaling out, means adding more machines to a pool and sharing the work among them.

Example: going from one web server to five.

Scaling up is simple, and it is a fine choice while traffic is low. But it has two limits. There is a ceiling, because a single machine cannot grow forever. And there is no safety net: if that one machine dies, everything stops.

So large systems scale out. Scaling out needs something to share the work.

Load balancer

A load balancer is a part that receives all incoming requests and spreads them across a group of servers. Users talk only to the load balancer, and the servers behind it are hidden.

Example: a receptionist who sends each new visitor to a free desk.

Two problems are solved at once:

  1. Failure. If one web server goes offline, the load balancer stops sending traffic to it. Users do not notice.
  2. Growth. If traffic grows, add servers to the group. The load balancer starts using them at once.

For safety, the servers behind a load balancer use private addresses, which cannot be reached from the internet. Only the load balancer is public.

A worked example with made-up numbers. One web server handles 200 requests per second. Peak load is 500 per second. Two servers give 400, which is not enough. Three give 600, which is enough, but if one fails the other two give 400 and drop requests. Four servers give 800, and with one down they give 600, so the site survives one failure at peak. The count is chosen from the peak and the failure plan.

Check yourself:

  • Why is one very big server a poor choice for a large site?
  • How many servers are needed for a peak of 900 requests per second at 200 each, surviving the loss of one?