Part 5 of 8 · 3 min read
Key-value and wide-column stores
Models built for fast lookup by key and for huge, sorted write streams.
Think of a coat check. Hand in a ticket number and receive exactly one coat. The attendant does not need to know what is in the pockets, and no one asks "which coats are blue?". The simplicity makes it fast.
Key-value store
A key-value store keeps one value for each key. It offers put(key, value) and get(key), and little else. The value is opaque to the database.
Example: put("session:9f3c", "{user: 7}") and later get("session:9f3c").
It is the right tool for sessions (Level 1, page 5), carts, caches (Level 3) and counters. It has no joins and asks no questions about the content of values. Its strength is speed and the ease of spreading keys over many machines: the key tells which machine holds the value (Level 4).
A worked example. A session store holds 2 million sessions of 1 KB. A lookup by key takes 0.5 ms. At 30,000 requests per second, the store handles 30,000 lookups per second. There is no query planner, no join, and nothing to tune except memory and copies.
The wide-column model grows out of Google's Bigtable paper, which the book mentions as the origin of Cassandra and HBase. It looks like a table, but it differs in important ways.
Wide-column store
A wide-column store keeps rows ordered by a row key. Each row can have many columns, and different rows can have different columns. Columns are grouped into families that are stored together.
Example: the row key sensor7#2026-11-12 has one column for each minute of the day, with the temperature as the value.
The two ideas that matter:
- Sorted by row key. Rows with nearby keys are stored near each other. A range scan over a key prefix reads one stretch of disk.
- Sparse columns. A row only stores the columns it has. A row with 3 values and a row with 3,000 values can share a table.
A worked example, a time series. A fleet of 10,000 sensors reports one reading each per second.
- Writes: 10,000 per second, or 864,000,000 per day.
- Row key:
sensor id + date. All readings for one sensor on one day sit in one row. A day has 86,400 columns, one per second. - Read "all readings of sensor 7 on 12 November" is one row read.
- Read "sensor 7 for the whole week" scans 7 adjacent rows.
This model supports very high write rates, since it is built on the log-structured storage that module 4 explains. It does not support joins, and questions that were not planned in the row key are slow. The row key must be designed around the main question. This is called designing from the query.
| Key-value | Wide-column | |
|---|---|---|
| Lookup by exact key | Yes | Yes |
| Range over keys | Usually no | Yes, sorted by key |
| Structure inside the value | None | Columns |
| Typical use | Sessions, caches, counters | Time series, activity logs, large write streams |
Read it as
Key-value answers "give me this one". Wide-column answers "give me this stretch, in order".
Check yourself:
- Why is the row key the most important design choice in a wide-column store?
- Name two uses that suit a key-value store.