Part 7 of 8 · 5 min read
Transactions and reports
What breaks when the data is apart, and how to limit the damage.
Transactions, reports and the cost of mistakes
A transaction is a group of changes that all happen or none do. In a single schema, creating an order and adding a row for the warehouse to pick can be one transaction. Once the schemas are apart, this safety is gone. If the order insert works and the picking insert fails, the data is out of step. BM gives three options.
| Option | How it works | The cost |
|---|---|---|
| Try again later | Accept the order, queue the picking insert and retry. The system becomes consistent at some later time (eventual consistency, Level 4) | Only works if a retry will fix it |
| Abort the whole operation | Undo the order with a compensating transaction, for example a delete | Which service holds that logic? What if the undo also fails? With 3 to 5 steps it becomes hard to follow |
| Distributed transaction | A transaction manager asks every participant to vote, then tells all to commit or all to roll back (two-phase commit) | Everyone waits for the manager. A crash blocks. Locks hurt scaling. Hard to get right |
BM's advice, in order: ask whether the operation really needs to be a single transaction. Many can be separate local transactions that settle later. If the state truly must be kept consistent, try hard not to split it. If it must be split, give the transaction a name and a body in the model, for example an "in-process order", so there is a place for the logic of undo and monitoring. This matches the saga idea of page 5.
Common mistake
Splitting the schema and finding out about transactions in production. List the multi-table changes first and decide for each whether it stays together, settles later, or needs a named process.
Reports are the other casualty. In a monolith, reports often run on a read copy of the one database, joining anything with SQL. That has costs: the schema becomes an interface for reporting, and the database cannot suit both jobs. After the split, the data is in many places. BM lists approaches.
- Calls to services. Fine for a small dashboard. It does not scale for large reports, such as two years of purchases. APIs are not built to return everything, and the load can hurt the service.
- Data pumps. A program pushes data from a service's database into a reporting database. This is a rare accepted case of one program reading another's tables, because the same team builds, versions and deploys the pump together with the service. The reporting schema stays a hard-to-change interface.
- Events. Services announce changes, and a reporting system collects them. BM moves toward this in later chapters. It matches Level 6 on change data capture.
Cost of change. Moving code in a codebase is cheap and quickly reversed. Splitting a database, or untangling a coupled interface, is costly and hard to undo. BM's rule is to make mistakes where they cost least. The cheapest place is the whiteboard. Sketch the design, walk the main use cases (search a record, register, buy an album) across the proposed boundaries, and look for circular calls and chatty pairs. BM suggests class-responsibility-collaboration (CRC) cards: one card per service, listing what it is responsible for and who it works with.
BM ends with a root cause. Services grow too large partly because splitting is expensive: somewhere to run, a new stack to set up. So lower the cost of a new service with shared libraries, light frameworks and self-service platforms. Growing a service until it needs a split is fine. The task is to notice before the split becomes too costly.
Interview angle
Answer in your head first, then open it.
How would you move from a monolith to services?
Find seams by context, organise the code into packages, choose the first split by benefit, then split the data and later the code.
What is the strangler pattern?
A front layer routes calls to old or new code and moves them over one by one, so there is no big rewrite.
What do you do with a shared table?
Find the concepts mixed in it, split it, give each context its own copy, and reach other contexts' data through an interface.
What breaks when you split a database?
Transactions and joins. Choose between retry later, undo steps, or a distributed transaction, and prefer to keep tightly tied data together.
Check yourself: 1. Why does a retry-later plan only work for some operations? 2. Why is a data pump allowed to touch another service's tables? 3. Where is the cheapest place to make a design mistake?
Words you will meet
| Word | Meaning |
|---|---|
| Seam | A part of a codebase that can be treated on its own |
| Strangler application pattern | A front layer that moves requests from old to new code a piece at a time |
| Façade service | A new service that hides an old system behind a clean interface |
| Foreign key | A database rule that links a row in one table to a row in another |
| Database refactoring | A small change to a database design that keeps its meaning |
| Transaction | A group of changes that all happen or none do |
| Compensating transaction | A new change that undoes an earlier one |
| Two-phase commit | A vote, then a commit, run by a transaction manager |
| Eventual consistency | The data agrees at some later time, not at once |
| Data pump | A program that moves data from a service's database into a reporting database |
| CRC card | A card listing a class or service, its duties and its partners |