Part 5 of 6 · 3 min read
Exercises
Each level has three exercises. Do the easy ones first.
Easy
Easy 1: Sort the errors (15 minutes)
Classify as retry or not: a 500, a 401, a connection reset, a 429 with Retry-After: 10, a 422, a 404 for a tool endpoint you just deployed.
Done when: each has an answer and one reason, checked with is_retryable where it applies.
Easy 2: Compute the waits (15 minutes)
With base 0.5 s, cap 8 s and no jitter, list the waits before attempts 2 to 7. What is the total wait if all six retries are needed?
Done when: you list six numbers and a total, and confirm them with the code.
Easy 3: One in five (20 minutes)
A run makes 15 calls and each fails with probability 2%. What is the chance that at least one call fails? What if each call gets three attempts in total, and the failures are independent?
Done when: you compute both (about 26% and about 0.01%).
Hint
the chance of no failure is 0.98 ** 15. With three attempts a call fails only if all three fail, 0.02 ** 3.
Medium
Medium 1: Retry in the model call (60 minutes)
Wrap the module 1 FakeLLM.create with retry_call, adding a wrapper that makes it fail with a 503 on a seeded fraction of calls. Show the success rate with max_attempts of 1, 2 and 4 over 200 calls.
Done when: you have a table of three rates and the average number of attempts.
Medium 2: Per-attempt timeouts (60 minutes)
Add a timeout_per_attempt to the policy and a fake call that takes a given number of simulated seconds. Calls slower than the timeout raise TimeoutError_. Test that a slow-then-fast service succeeds on attempt two, and that total simulated time never exceeds the deadline, even when a call itself is slow.
Done when: both behaviours are tested with the fake clock.
Hint
advance the fake clock by the smaller of the call time and the timeout. Before each attempt, check how much of the deadline is left.
Medium 3: Persistent idempotency (60 minutes)
Store keys and results in a SQLite table instead of a dictionary. Test that a second IdempotentExecutor object opened on the same database file returns the saved result without running the action.
Done when: a test simulates a restart and passes.
Hint
INSERT OR IGNORE with a primary key makes the table the source of truth.
Hard
Hard 1: A resilient client (120 minutes)
Combine retry, breaker and fallback in one ResilientClient with a primary and a backup service. Simulate 1,000 requests against a primary that is down for requests 300 to 600. Report how many requests succeeded, how many went to the backup, how many calls the primary actually received, and the mean latency in simulated seconds, with and without the breaker.
Done when: you have two result rows, and the breaker clearly reduces primary calls and latency during the outage.
Hint
count calls inside the fake service, not in the client.
Hard 2: Test a lost reply end to end (120 minutes)
Build a small fake payment service with idempotency-key support and a fault injector that drops the reply to the first request for each key. Show that a client without keys double-charges and a client with keys never does, over 500 simulated payments.
Done when: a test counts charges for both clients and asserts 0 double charges for the keyed client.
Hard 3: Resume after a crash (120 minutes)
Make an agent loop that saves its message list after every step. Simulate a crash (an exception) at a random step on 100 runs, resume from the file, and check that the final answer equals an uncrashed run and that no tool with a side effect ran twice (derive idempotency keys from the run ID and step number).
Done when: 100 of 100 runs match and the side-effect count is exact.
Hint
derive the key as f"{run_id}:{step}", so a resumed step reuses the same key.