CSE Roadmap

Level 4 of 13 · about 3 weeks

Scaling data: replication and partitioning

Copies for safety and speed, splits for size.

Modules

Core: everyone needs it. Common: most jobs use it. Specialist: optional depth, safe to skip on a first read.

  1. 1

    Replication: leaders and followers

    Keeping copies of a database on several machines, and what happens when the leader fails

    Core 8 parts · about 3 hours

  2. 2

    Replication lag

    What users see when followers are behind, and the three guarantees that hide it

    Core 7 parts · about 3 hours

  3. 3

    Multi-leader and leaderless replication

    Writes accepted in several places: conflicts, quorums, sloppy quorums and hinted handoff

    Common 8 parts · about 4 hours

  4. 4

    Partitioning (sharding)

    Splitting a data set across machines by key range or by hash, and avoiding skew and hot spots

    Core 8 parts · about 3 hours

  5. 5

    Secondary indexes and rebalancing

    Searching by something other than the key across partitions, and moving partitions when nodes come and go

    Common 8 parts · about 3 hours

  6. 6

    Consistent hashing

    The ring, virtual nodes, and adding or removing servers while moving few keys

    Core 8 parts · about 3 hours

  7. 7

    Request routing

    How a client finds the node that holds a key, and who keeps the map up to date

    Common 7 parts · about 3 hours

  8. 8

    Choosing a shard key

    Worked examples for users, orders, messages and time series, with a checklist

    Core 8 parts · about 4 hours