Level 4 of 13 · about 3 weeks
Scaling data: replication and partitioning
Copies for safety and speed, splits for size.
Modules
Core: everyone needs it. Common: most jobs use it. Specialist: optional depth, safe to skip on a first read.
1
Replication: leaders and followers
Keeping copies of a database on several machines, and what happens when the leader fails
Core 8 parts · about 3 hours
2
Replication lag
What users see when followers are behind, and the three guarantees that hide it
Core 7 parts · about 3 hours
3
Multi-leader and leaderless replication
Writes accepted in several places: conflicts, quorums, sloppy quorums and hinted handoff
Common 8 parts · about 4 hours
4
Partitioning (sharding)
Splitting a data set across machines by key range or by hash, and avoiding skew and hot spots
Core 8 parts · about 3 hours
5
Secondary indexes and rebalancing
Searching by something other than the key across partitions, and moving partitions when nodes come and go
Common 8 parts · about 3 hours
6
Consistent hashing
The ring, virtual nodes, and adding or removing servers while moving few keys
Core 8 parts · about 3 hours
7
Request routing
How a client finds the node that holds a key, and who keeps the map up to date
Common 7 parts · about 3 hours
8
Choosing a shard key
Worked examples for users, orders, messages and time series, with a checklist
Core 8 parts · about 4 hours