CSE Roadmap

Core tier · about 4 hours · 7 parts

Part 7 of 7 · 3 min read

Exercises

Easy

Easy 1: Offsets by hand (20 minutes)

A partition has messages at offsets 0 to 9. Consumer A has saved offset 4. Consumer B has saved offset 8. List the messages each still has to read and give each consumer's lag.

Done when: two lists and two lag numbers.

Hint

The consumer at offset 4 still needs offsets 4 to 9, so its lag is 6.

Easy 2: Which partition (20 minutes)

A topic has 4 partitions and uses user id mod 4. Give the partition for users 5, 8, 13, 22 and 40. Which pairs of users share a partition?

Done when: five partition numbers and the shared pairs.

Hint

Remainders repeat every four numbers.

Easy 3: Group capacity (20 minutes)

A topic has 5 partitions. Each consumer handles 300 messages per second. Give the group capacity for 1, 3, 5 and 8 consumers.

Done when: four numbers and the reason the last two are equal.

Hint

A partition goes to one consumer, so consumers beyond 5 sit idle.

Medium

Medium 1: Fill time (30 minutes)

A broker disk holds 4 TB. Producers write 40 MB per second on average. How long until the oldest data is overwritten? A consumer is down for 36 hours. Does it miss data? What should retention be to survive a 3-day outage?

Done when: three answers with the arithmetic.

Hint

Divide capacity by write rate for seconds, then convert to hours and days.

Medium 2: Pick the key (30 minutes)

A taxi app sends events: ride requested, driver assigned, ride started, ride finished. Choose a key so that all events of one ride stay in order. Explain what could go wrong if the key were the city name.

Done when: a key choice, a reason and a hot partition scenario with made-up numbers.

Hint

A ride id puts all events of one ride in one partition.

Medium 3: Replay after a bug (30 minutes)

A consumer wrote wrong totals for six hours. Write the steps to repair using replay, including where the offset starts, where the new output goes, and how to confirm before switching.

Done when: a numbered plan of at least five steps.

Hint

The new consumer group writes to a fresh table, so the old one stays until the check passes.

Hard

Hard 1: Size a topic (45 minutes)

A topic must handle 30,000 messages per second. A consumer handles 2,000 per second. Messages are 1 KB each. Retention is 3 days. Find the minimum number of partitions, the daily storage in GB, the storage for 3 days, and the storage with 3 replicas.

Done when: four numbers with the working and one sentence on room for growth.

Hint

Partitions needed is total rate divided by per-consumer rate. Round up.

Hard 2: Queue versus log design (45 minutes)

A system must (a) send 2 million marketing emails over an hour using 40 workers and (b) keep the full sequence of account changes for billing, fraud and search. Choose queue or log for each and justify with order, replay, parallelism and slow messages.

Done when: two decisions, each with four arguments, and a note on whether one tool could serve both.

Hint

Emails are independent tasks. Account changes need order and replay.

Hard 3: Slow partition (50 minutes)

One partition of 12 has lag growing while the others are at zero. List four likely causes (hot key, slow message, stuck consumer, uneven partitioning) and a test for each. Describe a fix that keeps per-key order.

Done when: a table of four causes with test and fix, and a statement of what order the fix keeps.

Hint

Compare per-partition message rate, message sizes and consumer CPU.