Part 6 of 7 · 3 min read
Costs and limits
Immutable history is powerful, but it has real costs.
Costs and limits
Think of a diary that can never be torn. It is wonderful for honesty and terrible if a page contains a secret you must remove by law.
DDIA lists the limits. How much immutable history can be kept depends on churn. A system that mostly adds data is easy. A system with many updates and deletes on a small dataset builds a huge history, and compaction and garbage collection become critical.
Deleting data. Regulations may require deleting a person's data after they close an account, or removing information that was wrongly stored. Appending a "deleted" event is not enough, because the data is still in the log. The book says you may need to rewrite history as if the data had never been written. Datomic calls this excision, and the Fossil version control system calls it shunning. Real deletion is hard because copies live in many places: storage engines, file systems, SSDs and backups. Deletion becomes a matter of making data harder to retrieve rather than impossible to retrieve.
A common industry technique for this is to keep personal data outside the events and refer to it by id, so that deleting the record deletes the person's data while the events remain. This is general knowledge and is not from the books used here.
Understanding the flow. The Building Microservices book warns that event-driven designs add complexity. It tells readers to be cautious about adopting them too eagerly, to have good monitoring and to use correlation IDs (page 1) so that requests can be traced across process boundaries.
Ordering and duplicates. Event consumers face the delivery issues of page 2. They must handle repeats. They may also see events in different orders unless they come from one partition (page 3).
Eventual consistency. Read models lag. Pages that need the freshest data must read from the write side.
A decision table:
| Question | Lean towards |
|---|---|
| Need a full audit trail or the ability to replay history? | Event sourcing |
| Many different read shapes from the same data? | CQRS with several read models |
| A simple app with one database and few readers? | Plain CRUD, with no events |
| Strict, immediate consistency between write and read? | Keep one model, or read from the write side |
| Strict data deletion rules? | Be careful: keep personal data outside the log |
Read it as
Event sourcing and CQRS are powerful but add moving parts. Many systems need only a part: events to tell other services what happened, with an ordinary database as the source of truth.
Interview angle
Answer in your head first, then open it.
What is event sourcing?
Store immutable events as the source of truth and derive current state from them. Snapshots speed up reads.
What is CQRS?
Separate write and read models. Read models are built from events and shaped for each query.
Choreography or orchestration?
Choreography for loose coupling, orchestration where one owner needs control and clear error handling. Choreography needs monitoring.
What are the downsides?
Stale read models, harder debugging, deleting data and operational complexity.
How do you show a user their own write immediately?
Read from the write side or show it locally while the view catches up.
Check yourself:
- Why is "just add a deleted event" not enough for a privacy deletion request?
- Give one case where plain CRUD is the better choice.
Words you will meet
| Word | Plain meaning |
|---|---|
| Command | A request that can be refused |
| Event | A fact that something happened |
| Event-based collaboration | Services react to each other's events instead of being told what to do |
| Orchestration | A central service drives the steps |
| Choreography | Services react to events with no central driver |
| Broker topology | Event flow spread through a broker with no mediator |
| Mediator topology | A central mediator controls the event workflow |
| Event sourcing | Events are the truth, state is derived |
| Snapshot | A saved copy of derived state to avoid replaying everything |
| Ledger | An append-only list of transactions |
| CQRS | Separate models for writing and reading |
| Read model | A view shaped for certain queries |
| Denormalised | Data copied into the shape needed to read it quickly |
| Excision | Truly removing data from an immutable history |
| Correlation ID | A tag that follows one request across services |