Part 2 of 8 · 2 min read
Requirements and numbers
A simple interface, a hard system.
Think of a huge wall of labelled lockers in a station. You give a label and get what is inside, or put something in under a label. The wall is so big that it spans many buildings, and some buildings are sometimes closed. The job is to keep every locker reachable and correct.
A key-value store holds pairs: a unique key and a value. The book lists examples such as Dynamo, Memcached and Redis. The value is treated as a block with no inner structure. Only two operations are needed:
put(key, value)stores a value under a key.get(key)returns the value for a key.
The book's scope for this design.
- Each pair is small, under 10 KB.
- It stores a very large amount of data.
- It is highly available: it answers quickly even during failures.
- It scales: servers are added or removed automatically with traffic.
- Consistency is tunable.
- Latency is low.
There is no perfect design. The book says each design balances reads, writes and memory, and balances consistency against availability.
Numbers (made up to show the idea). The store must hold 1 billion pairs of 5 KB on average. One copy is 1,000,000,000 × 5 KB = 5 TB. With 3 copies, that is 15 TB. A node with 1 TB of usable disk would need 15 nodes, and with a third left free for growth, about 20 nodes. For load, suppose 200,000 reads and 50,000 writes per second. Spread over 20 nodes, each handles about 10,000 reads and 2,500 writes per second for its share, plus copying work. This is easy for one node, which shows the main problem is data size and failures, not raw speed.
Key-value store
A key-value store is a database that keeps pairs of a unique key and a value, and offers lookup and storage by key.
Example: the key user:42:last_login with the value 2026-11-15T10:00:00Z.
Check yourself: 1. What two operations does the store offer? 2. How many nodes does 15 TB need if each node has 1 TB usable? 3. Why is raw speed not the main problem in the example?