CSE Roadmap

Common tier · about 4 hours · 7 parts

Part 5 of 7 · 3 min read

CQRS and read models

Write in the shape that suits writing, and build separate views shaped for reading.

Think of a library. Books arrive in the order they were bought, and are recorded in an accession book. Readers do not search that book. They use a catalogue by author, another by subject, and a shelf map. The catalogues are built from the accession book and updated as books arrive.

DDIA explains that with an event log, you can derive several different read-oriented views from the same events. This works like having several consumers of a stream: one consumer builds a search index, another builds a table for a dashboard, another builds a cache. The book names the idea command query responsibility segregation (CQRS).

CQRS

CQRS (command query responsibility segregation) means using one model to handle writes (commands) and separate models to answer reads (queries).

Example: orders are written as events. A search service builds an index of orders for lookup, and a reporting service builds a table of sales per day. Each is a read model.

Read model (view)

A read model is a copy of data shaped for a particular kind of question and built from the events or the primary data.

Example: a table "orders by customer, newest first", built by consuming order events.

The book's argument: the traditional approach to schema design rests on the fallacy that data must be written in the same form in which it will be queried. If you separate the write form from the read form, debates about normalised versus denormalised data lose much of their force. A read model may be fully denormalised, because the translation process keeps it consistent with the log. The book recalls Twitter's home timelines from Level 3 and Level 4: a cache of recent tweets by the people a user follows, highly denormalised, since a tweet is copied into every follower's timeline. The fan-out service keeps the copies in step with new tweets and new follow relationships.

There are further gains:

  • A new feature, a new view. To present existing data in a new way, build a new read model from the log and run it next to the old one. The book says running old and new side by side is often easier than a complicated schema migration, and the old one can be switched off when no longer needed.
  • Simpler concurrency. Many multi-object transactions exist because one user action must change several places. If an event is a self-contained description of the action, the action needs just one write: appending the event to the log, which is easy to make atomic.
  • Debugging and recovery. With an append-only log of facts, it is easier to see what happened and to recover from a bad deployment than when code could overwrite data.

Now the main cost. The consumers that build read models are usually asynchronous. A user writes an event and may then read from a read model that has not yet applied it. The book names the problem "reading your own writes" from Level 4. Fixes:

  1. Update the read model in the same transaction as the event append. This needs both in one storage system, or a distributed transaction.
  2. Let the client wait or read from the write side for a moment after its own write.
  3. Show the result of the write in the client's own screen while the view catches up. This is general UI practice, not from the books.

A worked example (made up). A user posts a comment. The event reaches the log at 10:00:00.000. The comment-count view is updated at 10:00:00.400. If the user reloads at 10:00:00.200, the count is one short. The page can show the comment from the client's own state to hide the gap.

Event arrives View updated Gap
10:00:00.000 10:00:00.400 400 milliseconds of staleness

If the view lags by minutes during a backlog (page 1), the gap is minutes.

Check yourself:

  • Why is a fully denormalised read model acceptable in CQRS?
  • What problem does an asynchronous read model cause for the user who just wrote?