CSE Roadmap

Core tier · about 3 hours · 8 parts

Part 6 of 8 · 2 min read

Stale data and the consistency gap

Why a cache and its database can disagree, and what limits the difference.

Think of a printed timetable in a station. The railway changes a train time, but the posters are not replaced until tomorrow. Anyone who reads the poster today is misled for a few hours. The railway accepts this for most trains and corrects urgent changes on the loudspeaker.

The cache holds a copy, and the source can change after the copy was made. The book "System Design Interview" explains the root of the problem: operations on the data store and on the cache are not in a single transaction, so the two can differ. Across several regions the difficulty grows. The book "Understanding Distributed Systems" frames the cache as best-effort replication and notes that an out-of-process cache shows every client the same version, which reduces inconsistency, even though that version may be out of date.

Ways the gap appears:

  1. Normal lag. The data changed and the TTL has not ended yet.
  2. Failed invalidation. The delete after a write did not reach the cache.
  3. Races. The reader and writer interleave badly (module 2).
  4. Several caches. An in-process cache on each server holds a different version.
  5. Replica lag. The cache reloads from a read replica [a copy of the database that serves reads, covered in Level 4] that has not yet received the write.

What bounds the gap:

  • A TTL is the upper bound on how long a wrong value can stay.
  • Invalidation after writes narrows the gap for most cases.
  • Versioned keys. Include a version number in the key, profile:42:v17. A change makes a new key, so no reader can get the old value under the new key.
  • Read-your-writes handling. After a user changes something, read their own data from the database for a short time, so they see their own change at once.

A worked example of the read-your-writes need. A user changes a display name. With a TTL of 5 minutes, the cache returns the old name to this user for up to 5 minutes. The user thinks the save failed and tries again. Reading the user's own profile directly from the database for 10 minutes after any change by that user removes the confusion. Other people may see the old name for up to 5 minutes, which is acceptable.

A note on cost versus value. Strict consistency would mean no caching, or caching with complicated protocols. The choice is made per kind of data: show the maximum acceptable age for each.

Data Maximum acceptable age
Static content (logos) Days
Product description Minutes
Price shown on a listing 1 minute
Price at payment 0 (read the database)
Stock count at payment 0

Check yourself:

  • A cache has a TTL of 10 minutes and the invalidation after writes fails once. What is the longest time the old value can stay?
  • What does a versioned key do that a TTL does not?