Part 4 of 8 · 3 min read
Synchronous and asynchronous copies
Whether the leader waits for followers before it tells the client the write succeeded.
Think of a manager who signs off a purchase. In one office, the manager waits until the assistant confirms the change is in the second ledger. In another, the manager says "done" at once and the assistant catches up soon after. The first is safer and slower. The second is quicker and risks losing a change if the manager's ledger burns down before the assistant writes it.
Synchronous and asynchronous replication
In synchronous replication, the leader waits for the follower to confirm that it received the write before reporting success to the client. In asynchronous replication, the leader sends the change and does not wait for a reply.
Example: a profile update is confirmed to the user only after follower 1 has the change (synchronous). Follower 2 gets it a moment later (asynchronous).
The book explains the trade-off.
- Synchronous advantage. The follower is certain to hold an up-to-date copy. If the leader fails suddenly, the data is still available there.
- Synchronous disadvantage. If the synchronous follower does not respond, because it crashed or the network failed, the write cannot go ahead. The leader must block all writes until it returns.
- Therefore it is impractical to make all followers synchronous: any one node outage would stop the whole system.
- In practice, one follower is synchronous and the others are asynchronous. If the synchronous one fails or is slow, an asynchronous one is made synchronous. There are always at least two nodes with the latest data: the leader and one follower. This is called semi-synchronous replication.
- Fully asynchronous. If the leader fails and cannot be recovered, writes not yet copied are lost. A write confirmed to the client is then not durable. The advantage is that the leader can keep taking writes even if all followers are behind.
The book notes that the weakening of durability may sound like a bad trade-off, but asynchronous replication is widely used, especially when there are many followers or they are spread across the world. Normally followers apply changes in under a second, but there is no guarantee. They can fall behind by minutes during recovery, near maximum load or on network faults.
A worked example. A leader takes 500 writes per second. Replication to a follower in another region takes 80 ms (made up).
| Mode | Time to confirm a write | Writes lost if the leader dies at a random moment |
|---|---|---|
| Asynchronous | 2 ms (local only) | up to 500 × 0.08 = 40 writes |
| Semi-synchronous, follower in the same data centre (1 ms) | 3 ms | 0 as long as the synchronous follower survives |
| Synchronous to the far region (80 ms) | 82 ms | 0 |
The far synchronous option makes every write 41 times slower than asynchronous. Many systems pick semi-synchronous with a near follower, plus asynchronous followers farther away.
The book also mentions research on variants that avoid data loss but still perform well. For example, chain replication, a variant of synchronous replication, is used in Microsoft Azure Storage.
Common mistake
Believing that "the client got a success reply" means "the data is safe on several machines". With asynchronous replication the reply can come before any follower has the write.
Check yourself:
- Why is it impractical to make every follower synchronous?
- A leader takes 800 writes per second and replicates asynchronously with 50 ms lag. How many writes are at risk?