Part 4 of 5 · 4 min read
Exercises
Easy
Easy 1: Run and read (20 minutes)
In code/mini-stop, run pytest and python demo.py. From section 1 of the demo, list for each stuck run (loop, thrash, wander, storm, jitter) which single guards stop it and which never do.
Done when: you give a small table with a mark per guard and say which guard stops all five.
Hint
the no-progress detector stops all five; the repeat detector misses two.
Easy 2: Which guard? (15 minutes)
For each failure, name the guard that would catch it fastest and one that would catch it late: (a) the model asks for the same file 30 times; (b) a web page returns 200,000 words; (c) a database query hangs; (d) the model searches 40 different phrasings and none returns anything.
Done when: each failure has a fastest guard and a backstop, with one sentence of reason.
Hint
(b) is caught by a word budget before a step limit; (c) needs a deadline inside the call.
Easy 3: Write the ending (15 minutes)
A research agent is stopped by its word budget after finding 4 of 7 sources. Write the message it returns to the user, including what it did, what it did not do and what the user can do next.
Done when: the message has a status, what was found, what is missing and a next step.
Hint
"I did not do" is as important as "I did".
Medium
Medium 1: A budget that passes down (60 minutes)
Add a parent run that starts two sub-runs and gives each half of its remaining word budget. Make the parent stop when the combined spend reaches its limit. Test with one sub-run that loops.
Done when: a test shows the looping sub-run is stopped by its share and the parent still returns the other's result.
Hint
pass a budget object, not a number, so both sub-runs draw from it.
Medium 2: A cycle detector for longer patterns (60 minutes)
Extend periodic to patterns of up to 5 steps and add a thrash5 scenario where five calls take turns. Show the old detector misses it and the new one stops it, and that no healthy scenario is stopped.
Done when: a test shows both results and the poll scenario still finishes.
Hint
for period p, compare tail[i] with tail[i % p], as periodic does for 1 to 3.
Medium 3: Tune the limits from data (90 minutes)
Generate 200 normal runs with needed steps from a long-tailed distribution (a few need 20+). Set the step limit at the 95th percentile times 1.5, measure how many normal runs it kills, and compare with the 99th percentile times 1.5.
Done when: you report the kill rate and the stuck-run waste for both settings and choose one with a reason.
Hint
sort the needed steps and take the value at index 0.95 x length.
Hard
Hard 1: A real agent with guards (150 minutes)
Wrap your Level 5 agent's loop with the five guards. Make a test task that makes the model loop on purpose (for example by removing the tool it needs) and show each guard stopping it in turn by switching the others off. (I did not run this, so your numbers are the ground truth.)
Done when: you have a table of guard against steps and words at stop, and every stopped run leaves a report with a partial answer.
Hint
keep a list of trace rows per run; every guard should write its name into the status.
Hard 2: Real cancellation (150 minutes)
Build a tool that runs a subprocess (a long script) and respond to a cancel flag by terminating it, waiting a grace period and then killing it. Prove with tests that a cancel arrives within a set time and that a half-written output file is not left behind.
Done when: tests cover cancel before start, during the process, and after it ended, and no stray process remains.
Hint
send a terminate signal first, wait, then kill; delete or write files atomically (temporary name, then rename).
Hard 3: False stops in the wild (120 minutes)
Collect 30 traces of successful runs from your agent. Replay them through your repeat and no-progress detectors at several thresholds and count how many good runs would have been stopped. Choose thresholds with a false-stop rate under 1%.
Done when: a table of thresholds against false stops exists, the chosen thresholds are justified, and one real wait or retry pattern that needed an exemption is described.
Hint
replay needs the recorded keys of each step; module 7 shows how to store them.