Part 5 of 10 · 2 min read
Copy the database
A leader that takes writes, followers that serve reads, and what happens when one fails.
The web tier now has no single point of failure. The data tier still does: one database.
Single point of failure
Any part whose breakdown stops the whole system. The short form is SPOF.
Example: a single database machine behind many web servers.
The common fix is replication: keep copies of the data on several machines.
Replication
Replication means keeping the same data on several machines, so that if one is lost another has it. One machine, the leader (also called the primary), takes all writes. The followers (or replicas) copy its data and answer reads.
Example: one leader database and three follower databases, with the leader copying every change to each follower.
Most applications read much more than they write, so a system often has several followers per leader. Benefits:
- Speed. Reads are spread over followers, so more queries run in parallel.
- Safety. If one machine is destroyed, for example by a disaster, the data survives elsewhere.
- Availability. If one database is offline, another answers.
What happens on a failure:
- A follower fails: reads go to the other followers, or to the leader for a while. A new follower replaces it.
- The leader fails: a follower is promoted to leader. In real systems this is delicate, because a follower may be slightly behind. Missing changes must be recovered, and this can be hard.
The full flow now: a user gets the load balancer's address from DNS. The load balancer sends the request to a web server. The web server reads from a follower and sends changes to the leader.
Common mistake
Assuming a follower always has the latest data. Copying takes time. A user may write, then read from a follower that is a moment behind, and not see their own change. Level 4 explains how to handle this.
Check yourself:
- Which database takes writes, and which take reads?
- Why does promoting a follower need care?