CSE Roadmap

Core tier · about 4 hours · 8 parts

Part 3 of 8 · 2 min read

Users

The simplest case: records read and written by their own id.

Think of a bank that files each customer's folder in a cabinet chosen by the customer number. To fetch a customer, the clerk goes to one cabinet.

Case: a user table with user_id, name, email, settings. Access: read and update a user by id; log in by email; list users by sign-up date for a report.

Choice: shard by hash of user_id.

  • Even data and load: hash spreads ids evenly.
  • "Get user 812" and "update user 812": one shard.
  • Stable: the id never changes.

Two queries do not name the key:

  1. Log in by email. The system has an email and needs the user id. Options: a separate lookup table (email → user_id), itself sharded by email. Or a global index on email (module 5). The lookup table is a small second sharded store, written whenever a user signs up or changes email.
  2. List by sign-up date for a report. A scatter/gather over all shards, acceptable for a rare report, or an analytics copy of the data in a warehouse (Level 2, module 5).

A worked example. 200 million users, 1 KB each: 200 GB of profiles. A shard holds 25 GB: 8 shards, or better, 64 partitions so that growth is easy (module 5). Logins: 3,000 a second. Each login does a lookup by email (1 shard of the email table) and then a read by id (1 shard of the user table): 2 shard calls. If the email table were not separate, a login would be scatter/gather over 64 partitions: 64 calls per login, 192,000 calls a second.

Query With user_id key only With email lookup table
Get user by id 1 shard 1 shard
Log in by email 64 shards 2 shards (lookup, then user)
Sign-up report 64 shards 64 shards (use a warehouse)

The celebrity case: a user with 50 million followers gets 200,000 profile reads a second. One shard cannot serve it. Cache the profile (Level 3, module 4) and, if needed, keep copies of that profile on several shards.

Hash or range? For users, hash. Range partitioning by user_id would be fine if ids were random, but sequential ids concentrate new sign-ups on the newest range (module 4).

Check yourself:

  • How is "log in by email" made cheap when the shard key is user_id?
  • Why is hash better than range for sequential user ids?