CSE Roadmap

Core tier · about 4 hours · 10 parts

Part 8 of 10 · 2 min read

Watch it, automate it, split the database

Logs, metrics and automation, and sharding, the last big step for data.

Logging, metrics and automation.

A small site on a few servers can get by with looking at files by hand. A large business cannot. Three tools become essential.

  • Logs are records of what happened, especially errors. Collect them in one searchable place.
  • Metrics are numbers over time. Host metrics (processor, memory, disk), group metrics (the health of the whole database tier or cache tier) and business metrics (daily users, revenue).
  • Automation means building, testing and releasing software with tools, not by hand. A common practice is checking every code change automatically, so that problems appear early.

Level 9 covers all three in depth.

Splitting the database: sharding.

Data keeps growing. Adding power to the database (vertical scaling) hits a ceiling. Some providers sell machines with several terabytes of memory, but there is still a limit, and cost rises steeply. The next step is horizontal: split the data over many machines.

Sharding

Sharding means splitting a large database into smaller pieces called shards, each on its own machine, and each holding a part of the data. A rule, the shard key, decides which shard holds which row.

Example: users 1 to 1,000,000 on shard A, 1,000,001 to 2,000,000 on shard B. Or a rule such as "user id modulo 4" picks one of four shards.

The shard key is the most important choice. A good key spreads data and traffic evenly. A poor key puts most data on one shard.

Three common problems of sharding:

  1. Resharding. When a shard is full, or the data is uneven, data must be moved to new shards. Level 4 explains a technique, called consistent hashing, that makes this cheaper.
  2. Hot spots. A very popular item, such as a celebrity account, can overload its shard even if the data is spread evenly.
  3. Joins. Joining tables across shards is hard and slow. A common fix is to copy some data so queries stay within one shard, at the cost of extra storage and care to keep copies in step.

Interview angle

Answer in your head first, then open it.

When would you shard a database?

Sharding is a heavy step. A good answer first uses cheaper steps (replicas, a cache, a bigger machine) and only then shards, with a clear reason and a named shard key.

Check yourself:

  • Why is the choice of shard key so important?
  • Why are joins across shards a problem?