Part 4 of 6 · 3 min read
Choosing a time to live
The trade-off between stale data and reloads, and how to avoid a wave of expiry.
Think of milk in a fridge with a date on it. Too short a date, and good milk is thrown away and bought again. Too long, and somebody drinks sour milk. The date is chosen by how fast the milk goes bad and how costly it is to buy more.
The book "System Design Interview" gives the advice directly: have an expiration policy, since data without one stays in memory for ever. It should not be too short, because the data then reloads from the database too often. It should not be too long, because the data can go stale.
Two quantities set the TTL:
- How stale may it be? This is a business limit. A weather page may be 10 minutes old. A stock count at checkout may be 0 seconds old.
- How costly is a reload? A key that costs 200 ms to rebuild is worth keeping longer than one that costs 1 ms.
A worked example. A key is read 100 times a second, and a reload costs 50 ms of database time.
| TTL | Reloads per second | Database time per second | Longest staleness |
|---|---|---|---|
| 1 second | 1 | 50 ms | 1 second |
| 10 seconds | 0.1 | 5 ms | 10 seconds |
| 60 seconds | 0.0167 | 0.83 ms | 60 seconds |
| 600 seconds | 0.00167 | 0.083 ms | 10 minutes |
Going from 1 to 10 seconds cuts the database cost tenfold. Going from 60 to 600 seconds saves less than 1 ms per second per key. The gain flattens quickly. With one key the choice hardly matters. With 2,000,000 keys, it matters a lot: at a 1-second TTL, if they were all hot, the reloads would be a flood.
A rule of thumb: choose the longest TTL the business can accept, and add active invalidation (module 2) so that important changes do not wait for the TTL.
A common trap is expiry that arrives all at once. Suppose 500,000 entries are filled at start-up with a TTL of exactly 3,600 seconds. At the 3,600th second, all of them expire together, and the next wave of reads goes to the database. To avoid this, add jitter: a random extra time to each TTL. This is a standard practice, not drawn from the supplied books.
Jitter
Jitter is a small random amount added to a TTL so that entries filled at the same moment do not expire at the same moment.
Example: a TTL of 3,600 seconds plus a random number from 0 to 360 gives expiry times spread across 6 minutes.
With 500,000 entries, a spread of 360 seconds turns a spike of 500,000 reloads in one second into about 500,000 ÷ 360 ≈ 1,390 reloads a second.
TTL is also used for data that must disappear. A one-time code with a 5-minute TTL cleans itself away. A login session can have a sliding expiry that is renewed at each use. Those uses are about correctness, not freshness.
Check yourself:
- A key costs 100 ms to reload and is read 20 times a second. How many reloads per second at a TTL of 5 seconds, and how much database time?
- What is the purpose of jitter?