CSE Roadmap

Core tier · about 3 hours · 6 parts

Part 2 of 6 · 3 min read

Two ways to fill a cache on a read

Who fetches the missing item: the application or the cache.

Think of a school library with a desk at the front. In one school, the student checks the desk shelf, and if the book is not there, walks to the main archive and then leaves a copy at the desk. In another school, the student asks the librarian for the book, and the librarian fetches it from the archive when needed. The student sees only the desk. The result is the same, but the work is split differently.

The book "Understanding Distributed Systems" describes both ways. When a miss occurs, the missing item has to be requested from the remote dependency and the cache updated. This can happen in two ways:

  • The client gets an "item not found" answer from the cache, asks the dependency itself, and updates the cache. The cache is then called a side cache.
  • If the cache is inline, it talks to the dependency directly, and the client only ever talks to the cache.

Cache-aside (side cache)

In cache-aside, the application code checks the cache first. On a miss, the application reads the database and writes the value into the cache itself.

Example: v = cache.get(key); if v is empty, v = db.read(key), then cache.set(key, v).

Read-through (inline cache)

In read-through, the application asks only the cache. On a miss, the cache itself loads the value from the database, stores it and returns it.

Example: the application calls cache.get(key). The cache holds the code that knows how to read the database.

A note on names. The book "System Design Interview" describes the first flow (check the cache, then the database, then store in the cache) in its scale-from-zero chapter and calls it a read-through cache. Other sources call that same flow cache-aside, since the application does the work. The words are used loosely. In an interview, describe who does the loading, and the name matters less.

A worked example with 1,000 reads of the same key, with a cache that is empty at the start:

  1. The first read misses. The database is read once and the value stored.
  2. The other 999 reads hit. The database is read 0 more times.
  3. Total database reads: 1 instead of 1,000.

The patterns differ in where code lives:

Cache-aside Read-through
Who loads on a miss The application The cache
Application code Handles both cache and database Talks to the cache only
Cache must know the database No Yes
If the cache is down The application can still read the database Reads fail, unless there is a fallback
Typical use A separate cache server such as Memcached A cache library or a cache product with a loader

An inline cache has one more advantage, noted in the book: because the cache does the fetching, it can return an entry whose time to live has passed when the database is unavailable, instead of an error. That keeps the service up with older data.

Cache-aside is the usual choice in system design answers. It is simple, it works with any cache server, and a dead cache costs speed but not correctness.

Check yourself:

  • In which pattern does the application code contain the database read on a miss?
  • Why does a cache-aside design survive a cache outage better than a read-through design?