Part 3 of 8 · 2 min read
Leaders and followers
How one machine takes the writes and the others copy them.
Think of a head clerk and several assistants, each with a copy of the ledger. Customers who want a change go to the head clerk, who writes it in the ledger and then sends the same change to every assistant. Customers who only want to look something up may ask any assistant. The change goes in one place first and is copied out.
Leader-based replication
In leader-based replication, one replica is the leader (also called primary or master). All writes go to the leader. The other replicas are followers (also called read replicas, secondaries or hot standbys). The leader sends each change to the followers in a replication log, and every follower applies the changes in the same order as the leader.
Example: a shop's database has 1 leader and 3 followers. Orders are written to the leader. Product pages are read from the followers.
The book lists where this is built in: PostgreSQL, MySQL, Oracle Data Guard, SQL Server, MongoDB. It is also used in message brokers such as Kafka and RabbitMQ, and in some network file systems.
Reads can go to the leader or any follower. Writes only go to the leader. Followers are read-only from the client's point of view.
Reads scale well this way for the common web pattern of mostly reads and few writes. But a write is processed by every replica. A follower must apply all the writes, so adding followers does not reduce write work per machine.
A worked example. Each machine can do 10,000 operations per second. The workload is 2,000 writes and 30,000 reads per second.
- Every replica applies all 2,000 writes. That leaves 10,000 − 2,000 = 8,000 operations per second per replica for reads.
- Reads needed: 30,000. Replicas needed: 30,000 ÷ 8,000 = 3.75, so 4 replicas.
- One leader and 3 followers. Writes: 2,000 per second, all on the leader. Reads: about 7,500 per second per machine.
- Doubling the writes to 4,000 per second leaves 6,000 for reads per replica, and 30,000 ÷ 6,000 = 5 replicas.
The writes limit what one leader can take. If the writes alone exceed 10,000 per second, no number of followers helps. Splitting the data across leaders (partitioning) is the answer to that.
Read it as
One place decides the order of writes. Many places answer reads. Followers copy, they never decide.
Two points on how reads are routed:
- The application or a proxy sends writes to the leader and reads to followers. A load balancer (Level 1, module 3) can spread the read traffic.
- A read that must see the newest data may have to go to the leader (module 2).
Check yourself:
- One server handles 5,000 operations per second. A workload has 1,000 writes and 12,000 reads per second. How many replicas are needed?
- Which node accepts writes in leader-based replication?