Part 3 of 4 · 2 min read
Exercises
Easy
Easy 1: Run it (30 minutes)
Run python -m pytest -q and python run_all.py in code/mini-deploy.
Done when: you report missed and falsely blocked changes for the gate at 200 cases.
Hint
the "point" rule is the flaky one; the "significant" rule is the underpowered one.
Easy 2: Lint your Dockerfile (30 minutes)
Run lint_dockerfile on a real Dockerfile.
Done when: you fix each issue found or explain why it does not apply.
Easy 3: Secret audit (30 minutes)
Run a real scanner (gitleaks or trufflehog) on your repository history.
Done when: you list findings and what you rotated.
Medium
Medium 1: CI pipeline (4 hours)
Write a CI workflow that lints, tests, runs a small eval set, builds an image and records its digest.
Done when: a failing eval blocks the merge.
Medium 2: Paired gate (3 hours)
Change the gate to a paired comparison on the same cases.
Done when: you show higher power at the same false-block rate for the same eval size.
Hint
compute the per-case difference and test its mean.
Medium 3: Expand-contract (3 hours)
Plan a column rename as three releases.
Done when: each release can be rolled back with the previous code still working.
Hint
release 1 adds the new column and writes both; release 2 reads the new column; release 3 drops the old one.
Hard
Hard 1: Canary with rollback (8 hours)
Deploy two versions on a container platform with a health check that rolls back automatically. (I did not run this, so your numbers are the ground truth.)
Done when: a deliberately bad release is rolled back without a human and you report failed requests.
Hard 2: Deploy during runs (8 hours)
Deploy while long runs are in flight. (I did not run this.)
Done when: no run is lost or duplicated and you report how in-flight runs resumed.
Hint
stop consuming new jobs first (drain), then replace workers.
Hard 3: Rollback rehearsal (6 hours)
Roll back a release that included an additive migration, then one that included a drop. (I did not run this.)
Done when: you document what broke and the fix.