Part 2 of 7 · 3 min read
The layers on the path
Every place a copy can be kept between the user and the data.
Think of a letter from a head office to a customer. A copy may be held by the customer, by the local post office, by the regional depot, by the head office clerk, and in the archive. A request for the same letter can be answered by the nearest holder.
The book "Building Microservices" splits caching into three groups. In client-side caching the client stores the result and decides when to fetch a fresh copy. With proxy caching, a proxy sits between client and server, such as a reverse proxy or a content delivery network. With server-side caching, the server handles it, perhaps with a system such as Redis or Memcached, or a simple in-memory cache. The author notes that for every public-facing website he has worked on, the result was a mix of all three.
The layers, from the user inwards:
| Layer | Where it runs | What it holds | Typical read cost |
|---|---|---|---|
| Browser cache | The user's device | Files and responses for that user | 0 (no network) |
| CDN | Servers near the user | Static files and some pages, shared by many users | Tens of ms |
| Reverse proxy | In front of your servers | Responses for many users | A few ms |
| Application cache (in-process) | Inside each server | Objects, results of calculations | Microseconds |
| Application cache (shared) | A cache cluster | Objects, sessions, query results | About 0.5 ms |
| Database caches | Inside the database and the operating system | Pages of table and index data | Memory speed |
The database layer is easy to forget. The book "Designing Data-Intensive Applications" notes that even a disk-based storage engine may never read from disk if there is enough memory, since the operating system keeps recently used disk blocks in memory. Reading hot data from a table can already be a memory read before any cache of yours is added.
The CDN, from the book "System Design Interview", is a network of geographically spread servers used to deliver static content such as images, video, CSS and JavaScript. Level 1, module 4 covers it in full.
A request walks inwards until a layer can answer:
- The browser checks its own cache. A fresh copy is used with no network traffic at all.
- If not, the request goes to the CDN, which answers from a nearby server.
- If not, it goes to your reverse proxy or load balancer, and then to an application server.
- The application checks its in-process cache, then the shared cache.
- If both miss, it queries the database, which may answer from its own memory.
- The answer is stored on the way back in one or more of these layers.
Cache hierarchy (layers)
A cache hierarchy is a chain of caches on the path of a request. Each layer answers if it holds a fresh copy, and passes the request on if it does not.
Example: browser, CDN, application cache and database page cache, in that order.
Check yourself:
- In which layer does a hit cost no network traffic at all?
- Name two layers that are shared between many users and two that are not.