Part 5 of 7 · 2 min read
How stale can the total get
The worst case age of a response that passes through several caches, and how to bound it.
Think of a rumour passed through several offices, each of which keeps notes for a day before checking again. A change at the source can take a day to reach the first office, another day to reach the second, and so on. The last office may be several days behind.
The book "Building Microservices" gives the warning: the more caches between you and the source of fresh data, the staler the data can be, and the harder it is to say how fresh a value is. With several services in a call chain, the freshness of a final answer is hard to assess. The advice is to keep it simple, stick to one cache if possible, and think carefully before adding more.
The worst case adds up. If a response passes through caches with TTLs t1, t2, ..., a copy fetched at the moment before a change can be handed on by each layer, so the worst-case age at the user is about the sum of the TTLs.
A worked example. A price is cached with these TTLs:
| Layer | TTL |
|---|---|
| Browser | 60 seconds |
| CDN | 120 seconds |
| Shared cache | 300 seconds |
The price changes in the database at time 0. The shared cache refreshed the old price just before, so it can hand out the old price until second 300. The CDN may copy that old price at second 299 and keep it for 120 seconds, until second 419. A browser may fetch it from the CDN at second 418 and keep it for 60 seconds, until second 478. The worst case is therefore close to the sum of the three TTLs, 300 + 120 + 60 = 480 seconds, or 8 minutes.
Even though each layer looks harmless on its own, the total is 8 minutes.
Ways to bound staleness:
- Shorter TTLs in outer layers. The browser and CDN get short TTLs for changing data. Long TTLs are kept for versioned files.
- Purge the layers you control. After a write, delete the key in the shared cache and purge the CDN. The browser cannot be purged, so its TTL must stay short, or the URL must change.
- Versioned names or keys. A change makes a new name, so there is no old copy to be stale.
- Conditional requests. A short max-age plus an ETag makes layers check cheaply.
- Different rules per kind of data. A price on a listing may be 5 minutes old. A price at checkout is read from the database.
A worked example of the purge. After a price change, the application deletes the shared cache key, then calls the CDN's purge for that URL. The remaining staleness is the browser's 60-second TTL. Instead of 480 seconds, the worst case is 60 seconds.
Check yourself:
- Layers with TTLs of 30, 60 and 120 seconds. What is the worst-case age?
- Which layer cannot be purged by the server, and what does this mean for its TTL?