Part 4 of 7 · 3 min read
Event sourcing
Store the sequence of things that happened, and work out the current state from it.
Think of a bank statement. It lists each deposit and withdrawal. The balance at the bottom is worked out from the list. If a deposit was a mistake, the bank does not rub it out. It adds a correction line. The list is the truth, and the balance is a summary.
DDIA describes event sourcing as a technique from the domain-driven design community. The application's changes are stored as a log of events, and application logic is built on top of immutable events written to an event log. The event store is append-only, and updates or deletes are discouraged or forbidden. Events reflect things that happened at application level, not low-level changes to rows.
Event sourcing
Event sourcing is storing every change to an application as an immutable event, in order, and treating that log as the source of truth. The current state is computed from the events.
Example: an account is stored as "opened with 0", "deposited 100", "withdrew 30". The balance of 70 is computed, not stored as the truth.
The book compares event sourcing with change data capture (page 7). With change data capture, the application uses a database in the usual way, updating and deleting rows, and the changes are pulled out from the low-level log afterwards. The application does not know. With event sourcing, the application itself is designed around events.
The DDIA book gives an example of why the higher-level event is better. The event "student cancelled their course enrollment" expresses the intent of one action neutrally. The side effects, "one entry was deleted from the enrollments table and one cancellation reason was added to the feedback table", build in assumptions about how data will be used. If a new feature is introduced, "offer the place to the next person on the waiting list", it can be chained off the existing event.
Deriving state. An event log alone is not useful to a user, who wants the current cart and not a list of every change. So the application turns the log into state suited to display. The transformation must be deterministic, so replaying gives the same state each time.
A worked example (made up). A shopping cart.
| Order | Event | Cart after this event |
|---|---|---|
| 1 | Item "shoes" added | shoes |
| 2 | Item "socks" added | shoes, socks |
| 3 | Item "shoes" removed | socks |
| 4 | Item "hat" added | socks, hat |
The cart is "socks, hat". The log also keeps something the final cart does not show: the customer considered shoes and decided against them. DDIA points out that this can be valuable for analytics. A database that deletes the item on removal would lose it.
Snapshots. Replaying a long log for every read is slow. So applications save snapshots of the derived state from time to time, and replay only the events after the snapshot. The book stresses this is just an optimisation. The intention is that all raw events are kept and can be replayed at any time.
Compaction does not work the same way. In change data capture, an event holds the whole new version of a record, so only the latest event per key matters and older ones can be dropped (page 7). In event sourcing, events say what a user meant. Later events do not override earlier ones. "Deposited 100" and "deposited 50" both count. So the whole history is needed.
Ledgers. DDIA notes that accountants have used this for centuries. Transactions are written in an append-only ledger. Accounts are derived by adding them up. A mistake is corrected by adding a compensating entry, never by erasing, because the original may matter for an audit.
Common mistake
Putting facts that can still be refused into the event log, or letting a consumer change an event. An event is a fact. Corrections are new events.
Check yourself:
- Why can log compaction drop old change data capture events but not old event sourcing events?
- What is a snapshot for, and what does it not replace?