Part 3 of 8 · 1 min read
First designs
In-memory, then databases.
API. One call: POST /v1/wallet/balance_transfer with the source and destination accounts, an amount (as a string, never a float, as in Level 12 page 4) and a transaction id that is unique for each request.
Design A: in-memory key-value store. The balance of each user is a pair (user, balance) in Redis. One Redis node cannot take 1 million transfers a second, so users are spread over a cluster. Partition by hashing the key and taking the remainder after dividing by the number of nodes. The list of nodes lives in a configuration store such as ZooKeeper. A stateless wallet service checks the command, then updates the two nodes.
The flaw. The two updates go to two different nodes. If the service crashes after the first, the transfer is half done. Money is lost or created.
Design B: transactional databases. Replace each Redis node with a relational database partition. A transfer inside one partition is one atomic transaction. But when A and C are on different partitions, there are still two separate transactions. A restart between them leaves the same half-done transfer. A distributed transaction is needed.
A worked example (made up). A has $10 on partition 1, C has $0 on partition 2. A transfer of $1 deducts from A and commits on partition 1. The wallet service then crashes. A has $9 and C has $0. One dollar is gone.
Common mistake
Updating two accounts in two places with two plain writes and hoping nothing fails in between. Every crash window is a chance to lose money.
Check yourself: 1. Why is the Redis design not safe? 2. When is a transfer a single transaction? 3. What is the loss in the worked example?