CSE Roadmap

Core tier · about 4 hours · 7 parts

Part 6 of 7 · 3 min read

Choosing a guarantee

Match the promise to what a lost or repeated message would cost.

Choosing a guarantee

Think of three jobs: counting website hits for a rough chart, charging a credit card, and sending a "welcome" email. A lost hit does not matter. A double charge does. A double welcome email is only annoying.

The right question for each message type is: what does a loss cost, and what does a repeat cost?

Job Cost of a loss Cost of a repeat Sensible choice
Rough traffic chart Tiny Tiny At most once is fine
Exact order count Wrong totals Wrong totals At least once plus idempotent consumer
Charge a card Lost revenue, unhappy shop Customer charged twice At least once plus idempotency key
Welcome email Missed email Two emails At least once, repeat is acceptable
Update a cache entry with a full value Stale cache None, it sets the same value At least once, naturally idempotent

The Fundamentals of Software Architecture book makes a related point about replays. In an event-driven design, replaying events is a way to recover, but it is complex "especially if the events/requests being replayed aren't idempotent". This links delivery guarantees to the safety of recovering after a failure: the more operations are idempotent, the more freely a team can re-run work.

A practical checklist for a design review:

  1. Write down every consumer and ask: what happens if this message arrives twice?
  2. For each non-idempotent consumer, add an id check or change the operation to "set".
  3. Ask: what happens if this message never arrives? For critical ones, add retries and a dead letter queue (page 1).
  4. Test by delivering each message twice in a test environment.

Interview angle

Answer in your head first, then open it.

Can a queue guarantee exactly-once delivery?

No. A lost acknowledgment forces a resend. The usual answer is at-least-once delivery plus idempotent processing, which gives an exactly-once effect.

How do you make a consumer idempotent?

Use a unique message id and a table of processed ids, saved in the same transaction as the work, or use "set" operations, or store the offset with the write.

At-most-once or at-least-once?

Ask what a loss costs against a repeat. Money and inventory want at-least-once with idempotence. Telemetry can accept loss.

What does exactly-once mean in Kafka-style streams?

Atomic commit of output, state and position inside the framework, so the visible effect happens once.

Check yourself:

  • A fraud alert system must never miss an event but may alert twice. Which guarantee fits?
  • Why do external side effects, such as emails, still need care in an exactly-once framework?

Words you will meet

Word Plain meaning
Delivery guarantee A promise about how many times a message arrives
At most once Zero or one delivery. May lose
At least once One or more deliveries. May repeat
Exactly once One effect per message
Effectively-once Repeated work, but the effect shows once
Acknowledgment (ack) The consumer telling the broker it is done
Idempotent Safe to repeat, same result as once
Duplicate A second copy of a message that was already handled
De-duplication Dropping repeats, usually by message id
Checkpoint A saved position or state to resume from
Offset The position of a message in a log
Microbatching Cutting a stream into tiny batch jobs
Atomic commit Several effects happen together or not at all
Fencing Stopping a node that was thought dead from acting