CSE Roadmap

Common tier · about 3 hours · 7 parts

Part 6 of 7 · 4 min read

Caching writes and resilience

Caches that absorb writes, serve stale data in an outage, or fail fast.

Caching writes and resilience

Think of a shop that keeps a stack of copies of yesterday's newspaper. If the supplier fails to deliver today, the shop can still offer yesterday's paper. It is not today's news, but it is better than an empty rack.

Caching for writes. The book "Building Microservices" says reads are the usual use of caching, but a write-behind cache fits some cases: bursts of writes, or the same data written many times. Writes are buffered, perhaps batched, and flushed later to the main source. If the buffered writes are stored safely, they can even be queued during an outage of the downstream service and sent when it returns (module 2 covered the risk of losing buffered writes).

Caching for resilience. The same book describes serving stale data when the source is unavailable. With client-side caching, a client can use a cached copy if the downstream service is down. A reverse proxy can serve stale data too. For some systems, being available with old data beats returning no result, and that is a judgment call about the business.

It also gives a striking example. A team crawled the live site from time to time to produce a static version, which was served if the site was down. It was not as fresh as the live cache, but it kept a version of the site on screen during an outage.

A worked example. A news site with 30,000 reads a second loses its database for 10 minutes.

  • No stale serving: 600 seconds × 30,000 = 18,000,000 failed requests.
  • Serving stale pages from the CDN and shared cache with stale-while-error allowed: pages in cache (say 98 percent of requests) still work. Failed requests are 2 percent of 18,000,000 = 360,000.
  • The readers see pages up to a few minutes old during the outage, which is acceptable for news.

Hiding the origin. As the book describes (module 4), when the cache loses a large part of its content, requests should not rush to the origin. The cache is refilled in the background and the original request can fail fast. This keeps one failure from cascading.

Simplicity. The more layers, the more places for staleness, bugs and cost. A sensible order for a new system:

  1. HTTP caching with versioned file names for static files, and a CDN.
  2. A shared application cache for the most expensive and most-read data.
  3. An in-process layer for the few hottest keys.
  4. Further layers only when measurements show a need.

Interview angle

Answer in your head first, then open it.

Where would you cache in this design?

Walk the path from the browser to the database. Name the layers you use, the TTL of each, and what each saves. Say which layers you skip and why.

How stale can the data get?

Add the TTLs of the layers. Show how purging or versioned names cut it.

How do browsers and CDNs know how long to keep a response?

Cache-Control and Expires set the time. ETag with a conditional GET checks a held copy cheaply and returns 304 if it is current.

What reaches the database?

Multiply the miss ratios of the layers. Give the figure.

What if the source is down?

Serve stale copies for data where that is acceptable, and fail fast for the rest.

Check yourself:

  • Why is a static copy of a site useful during an outage?
  • Name the order in which you would add cache layers to a new system, and why.

Words you will meet

Word Plain meaning
Cache hierarchy A chain of caches on the path of a request
Client-side cache A cache held by the client, such as a browser
Proxy cache A cache between client and server, such as a CDN or reverse proxy
Server-side cache A cache run by the service, in memory or on a cache server
Cache-Control A header giving the rules and time for reusing a response
max-age The number of seconds a response stays fresh
Expires A header with the date and time after which a response is stale
ETag A label for one version of a resource
Conditional GET A request that returns the resource only if it changed
304 Not Modified A reply meaning the client's copy is still current
Purge Deleting a cached copy on request
Write-behind cache A cache that takes writes and sends them on later
Static copy A saved plain version of a site served during an outage