CSE Roadmap

Core tier · about 3 hours · 7 parts

Part 4 of 7 · 2 min read

Replicated, distributed and near caches

Three ways to arrange caches when data is spread over many processing units.

Think of a company with 10 branch offices. Each branch can hold a full copy of the price list, so every look-up is local, but every change must reach all 10 copies. Or the head office holds the one list and the branches ask by phone: always right, slower, and a problem if the line is down. Or each branch keeps its most-asked pages and phones for the rest.

The book "Fundamentals of Software Architecture" describes these in the setting of a space-based architecture, where each processing unit holds data in memory.

  • Replicated cache. Each processing unit holds a full copy of the cache, and the copies are kept in step by a replication engine.
  • Distributed cache. The data is held in a central cache, and processing units ask it.
  • Near-cache. A hybrid. The distributed cache is the full backing cache, and each unit holds a smaller front cache, a subset that is trimmed by an eviction policy. The front cache can be a most-recently-used or most-frequently-used cache, or can use random replacement when no analysis shows which is better.

The book's comparison:

Criterion Replicated cache Distributed cache
Optimised for Performance Consistency
Cache size Small (under 100 MB) Large (over 500 MB)
Type of data Relatively static Highly dynamic
Update frequency Relatively low High
Fault tolerance High Low

The book's advice is that a distributed cache always offers better consistency than a replicated one, since the data sits in one place. A replicated cache always gives better performance and fault tolerance. The choice depends on the type of data: highly consistent data, such as inventory counts, suits a distributed cache; data that changes little, such as reference data like product codes or descriptions, suits a replicated cache. Both can be used in the same system for different data.

A caution about the near-cache. The front caches are kept in step with the backing cache but not with each other, so different units are likely to hold different data. This produces uneven performance and, the book says, is why it does not recommend a near-cache for space-based architecture.

A worked example. A system has 20 processing units. The country list is 40 KB and changes twice a year. A replicated cache costs 20 × 40 KB = 800 KB in total and makes every read local. Inventory counts, 4 GB, change every second. A replicated cache would copy 4 GB to each of 20 units (80 GB) and send every update 20 times. A distributed cache stores them once (4 GB) and reads cost a network trip.

Check yourself:

  • Which data would you put in a replicated cache: the list of countries or live stock counts?
  • Why does a near-cache lead to different units seeing different data?