Part 6 of 8 · 3 min read
Time series
Writes that always arrive "now", and keys that avoid a hot shard.
Think of a record of the temperature at 1,000 weather stations, one reading per minute each. All the new readings are for the current minute. If readings are filed by time, today's cabinet takes every new reading while other cabinets sit idle.
The book "Designing Data-Intensive Applications" uses this example. A key range partition by timestamp puts all of today's writes on today's partition. The fix is to put the sensor name first and the time second, so that writes spread over many partitions.
Case: 5,000 devices send one reading every 10 seconds: 500 writes a second, 43,200,000 a day. Queries: the last hour of device 42, the average over a day for one device, a map of the latest value for all devices, and a monthly report over all devices.
Candidates:
| Key | Writes | "Device 42, last hour" | "Latest value of all devices" | "Month report, all" |
|---|---|---|---|---|
| Timestamp (range) | One hot shard | Many (scan the hour for all devices) | One shard (the latest) | Many, sequential |
| Device id (hash) | Even | 1 shard, but all history in one partition | All shards | All shards |
| (device_id, time) compound | Even | 1 shard, a short range | All shards | All shards |
| (device_id, day bucket) compound | Even | 1 shard (2 at day boundary) | All shards | All shards |
The compound key (device_id, day) as the partition key with time as the sort key is common. Each device-day is a bounded partition: 8,640 readings of 50 bytes = 432 KB. Without the day, one device's partition would grow for ever.
Other time-series ideas:
- Hot recent data, cold old data. Recent data is read most and kept on fast storage (and cached). Old data moves to cheaper storage in larger blocks, or is rolled up [summarised, for example 10-second readings become hourly averages], and the raw data is deleted after a set time.
- Batching writes. A device-day partition receives writes in order. The store appends them (the log-structured style of Level 2, module 4), which suits this pattern.
- Retention. Dropping a whole partition is cheap, but deleting row by row is costly. Day buckets make a 30-day retention a matter of dropping partitions older than 30 days.
A worked example of the hot shard. 20 partitions, 500 writes a second.
| Key | Writes per second on the busiest partition |
|---|---|
| Timestamp ranges (one per hour) | 500 (all) |
| (device_id, day) | About 25 (500 ÷ 20) |
A shard with 5,000 writes a second of capacity is far below the limit in the second case and is at 10 percent in the first. If the fleet grows by 10 times to 5,000 writes a second, the timestamp scheme would be at 100 percent on a single shard, while the compound key stays at 250 per shard.
Common mistake
Using the timestamp as the first part of the key for a stream of new data. All writes go to one shard whatever the number of nodes.
Check yourself:
- What is the size of one device-day partition if readings are 60 bytes every 5 seconds?
- Why does the bucket in the key make retention easier?