Part 6 of 12 · 2 min read
What the example found
Attack success rates on train and held-out families by defence, attempts needed at several true rates, and the CI gate's exit codes.
How to read the defence-comparison table: each row is a different defence applied to the same 270-attack grid; the columns split by whether the three phrasing families were used to design the defence.
| Target | ASR train families | ASR held-out families | Severity-weighted (all) |
|---|---|---|---|
| No defence | 20.7% | 22.3% | 21.9% |
| Blocklist learned from train attacks | 5.2% | 9.3% | 7.5% |
| Gate v1 (block email, delete, large refund) | 9.0% | 9.3% | 4.0% |
| Gate v2 (+ per-customer scope, refund cap 50) | 0.0% | 0.0% | 0.0% |
What to notice: the blocklist is the only row where the train and held-out columns disagree by a meaningful margin (5.2 vs 9.3). Both gates give nearly identical numbers in both columns, which is the table-level signature of a structural fix versus a wording-dependent one.
How to read the attempts-needed table: each row is a different assumed true success rate; the column is how many independent attempts are needed for a 95% chance of seeing at least one success at that rate.
| True success rate | Attempts for 95% chance of one success |
|---|---|
| 10% | 29 |
| 5% | 59 |
| 1% | 299 |
| 0.1% | 2,995 |
What to notice: the attempts needed roughly scale with the inverse of the rate. Going from 1% to 0.1% (a 10x smaller rate) needs almost exactly 10x more attempts. That is the arithmetic reason a thorough red-team suite run only a few hundred times can confidently rule out a 1% failure mode. But it says very little about anything rarer than that.
How to read the CI-gate table: each row is the same defence configuration as the first table, with the pass/fail rule applied: fail if the interval's lower bound exceeds 2% (or the severity-weighted rate exceeds 2%).
| CI rule: fail if the interval's lower bound exceeds 2% (or severity-weighted exceeds 2%) | Exit code |
|---|---|
| No defence | 1 |
| Blocklist | 1 |
| Gate v1 | 1 |
| Gate v2 | 0 |
What to notice: the blocklist still fails the CI gate despite its large headline improvement over "no defence". A drop from 22% to 7.5% severity-weighted is a real improvement, but it's nowhere near the 2% threshold. That is exactly the point of setting the gate on an absolute limit rather than on "did it get better.".
{{svg:cycle|Generate, attack, measure, fix, re-test with held-out attacks and a regression suite.}} {{svg:results|Attack success rates by defence on train and held-out attack families, with intervals and regression suite sizes.}}