Part 3 of 4 · 2 min read
Exercises
Easy
Easy 1: Run it (30 minutes)
Run python -m pytest -q and python run_all.py in code/mini-debug.
Done when: you can say why recall is circular.
Easy 2: Read ten traces (1 hour)
Print ten faulty traces and name the first abnormal span in each without looking at the label.
Done when: you agree with the label in at least eight.
Easy 3: Write the six beats (1 hour)
Write a 2-minute answer about a real bug from one of your projects using the six beats.
Done when: every beat has evidence, not just a conclusion.
Medium
Medium 1: Add a fault class (3 hours)
Add a ninth fault (for example a wrong tool for the task) to traces.py and a rule for it.
Done when: you report its precision, recall and what looks similar to it.
Medium 2: Tune a threshold (3 hours)
Choose a latency threshold for a cost ratio of 1 missed spike to 5 false alerts.
Done when: your choice follows from the sweep table and you can defend it.
Medium 3: Regression test (3 hours)
Turn a failing trace from your own agent into a replayable test that fails before your fix and passes after.
Done when: a deliberate reintroduction of the bug fails the test.
Hard
Hard 1: Overlapping faults (8 hours)
Generate traces with two faults at once and measure how localisation and detection degrade. (I did not run this, so your numbers are the ground truth.)
Done when: you report which pairs confuse the detectors.
Hard 2: Quality check (8 hours)
Add a model-based or human-labelled check for the fluent-wrong class and report its precision and recall with intervals. (I did not run this.)
Done when: the class has a measured recall above zero.
Hard 3: Real-trace study (8 hours)
Label 50 real failures from your agent, group them by cause and compute which share your rules could flag. (I did not run this.)
Done when: you have a Pareto table of causes and a plan for the top two.