Part 7 of 7 · 3 min read
Exercises
Easy
Easy 1: Label the guarantee (20 minutes)
Name the guarantee in each case: a fire-and-forget UDP metric; a queue that resends until acknowledged; a consumer that deletes a message before working on it; a consumer that works and then deletes.
Done when: four answers, each with the bad outcome that can happen.
Hint
Delete first loses messages. Process first repeats them.
Easy 2: Idempotent or not (20 minutes)
Decide for each: set email to a new address; add 5 to a counter; insert a row with a unique order id (second insert fails); send a push notification; delete a file by name.
Done when: five answers with one line of reasoning each.
Hint
"Set to value" operations and unique-key inserts survive repeats.
Easy 3: Count the duplicates (20 minutes)
A system delivers 50,000 messages. A crash between processing and acknowledging hits 0.2 percent of them. How many are processed twice? How many are lost if the system uses delete-first instead?
Done when: both numbers with the arithmetic.
Hint
0.2 percent of 50,000 is 100.
Medium
Medium 1: Fix a points service (30 minutes)
A rewards service handles "credit N points to account A" messages and is not idempotent. Redesign the message and the table so that repeats are harmless. Show the table columns and the check.
Done when: a message format, a table definition, and a step list for a repeated message.
Hint
Add a column for the purchase id and make it unique per account.
Medium 2: Crash timeline (30 minutes)
A consumer reads offsets 100 to 109, writes results to a database, and saves its checkpoint every 5 messages. It crashes after processing offset 107 and before saving the second checkpoint. Which offsets are processed twice? Show how storing the offset with each write prevents wrong results.
Done when: a timeline, a list of repeated offsets and a rule that skips them.
Hint
The last saved checkpoint covers offsets 100 to 104, so 105 to 107 are read again.
Medium 3: Atomic or idempotent (30 minutes)
A stream job reads clicks, counts them per minute, and writes counts to a database. Compare two fixes for duplicates after a restart: writing count and offset in one transaction, and writing "set count" with the minute as the key. State the assumptions each needs.
Done when: a table of two approaches with assumptions, costs and a recommendation.
Hint
The "set count" approach needs the same events to replay in the same order.
Hard
Hard 1: Payment webhook (45 minutes)
A payment provider calls your web address with "payment succeeded" events, and retries if you do not answer within 5 seconds. Design the handler so that an order is marked paid exactly once in effect, even if the provider sends the same event three times, concurrently.
Done when: a design with an id table, a transaction or unique constraint, and a walk-through of three simultaneous copies.
Hint
A unique constraint on the event id lets the database reject copies even when they run at the same time.
Hard 2: Exactly-once pipeline (45 minutes)
A pipeline reads orders from a log, enriches them, writes to a database, sends an email, and publishes to another topic. List each side effect and how to make it safe to repeat. Mark any that cannot be made idempotent and propose a compromise.
Done when: a table of effects with method, assumption and residual risk.
Hint
An email cannot be taken back. Record "email sent" with the order id before or with the send, and accept the small risk.
Hard 3: Choose per message (50 minutes)
A ride-hailing app sends messages for location pings, ride requests, fare charges, driver payouts, rating prompts and receipts. Assign a delivery guarantee and consumer design to each. Estimate, with made-up numbers, the volume of duplicates you would tolerate and the cost of each design choice.
Done when: a six-row table with guarantee, consumer design, tolerated duplicates and reason.
Hint
Location pings can be at most once. Money must be at least once with keys.