CSE Roadmap

Core tier · about 4 hours · 10 parts

Part 7 of 10 · 2 min read

Keep servers stateless, add data centres and queues

Three moves that let the system grow across the world and absorb slow work.

Stateless web tier.

Think of a bank with one clerk per customer. If customer A always has to go to clerk 1 because only clerk 1 holds the file, the queue at clerk 1 grows while others sit idle. If all the files are in a shared cabinet, any clerk can serve any customer.

Stateless

A server is stateless when it keeps no information about users between requests. Anything that must be remembered, such as login sessions, lives in a shared store outside the servers.

Example: a web server that reads a user's session from a shared store on every request, and keeps nothing itself.

A stateful server remembers data from one request to the next, such as a session. The load balancer must then send each user always to the same server, using "sticky sessions". That adds overhead and makes adding, removing or replacing servers hard.

With state moved out into a shared data store, any server can answer any request. The number of servers can then rise and fall with traffic. This is called autoscaling: adding or removing servers automatically based on load.

More than one data centre.

A data centre is a building of computers. As users appear around the world, one data centre is too far from many of them, and one fire or power cut could take the site down.

The fix is several data centres, with users sent to the nearest one by geoDNS, a DNS service that answers with a different address depending on where the user is. If one data centre fails, all traffic goes to the others.

Three problems follow:

  1. Sending traffic correctly. The tool must direct each user to the right data centre.
  2. Data in step. Each region has its own data. After a failover, a user may arrive in a region that lacks their data, so data is copied between regions.
  3. Testing and release. Every location must be tested, and automation keeps all copies of the software the same.

Message queue.

Think of a restaurant ticket rail. The waiter hangs an order on the rail and goes back to the tables. The cook takes tickets when ready. Neither waits for the other.

Message queue

A message queue is a waiting line, kept safe, for messages sent between parts of a system. A producer adds a message. A consumer takes it later and does the work.

Example: a photo app puts "resize this photo" jobs in a queue. Workers take jobs and resize photos.

A queue decouples the two sides: the producer can post even when the consumer is down, and the consumer can work even when the producer is quiet. Each side can be scaled separately. If the queue grows long, add workers. If it is mostly empty, remove workers.

Check yourself:

  • Why do sticky sessions make adding servers harder?
  • Which part sends users in Europe to a nearby data centre?