CSE Roadmap

Common tier · about 4 hours · 12 parts

Part 6 of 12 · 2 min read

What the example found

Attack success rates on train and held-out families by defence, attempts needed at several true rates, and the CI gate's exit codes.

How to read the defence-comparison table: each row is a different defence applied to the same 270-attack grid; the columns split by whether the three phrasing families were used to design the defence.

Target ASR train families ASR held-out families Severity-weighted (all)
No defence 20.7% 22.3% 21.9%
Blocklist learned from train attacks 5.2% 9.3% 7.5%
Gate v1 (block email, delete, large refund) 9.0% 9.3% 4.0%
Gate v2 (+ per-customer scope, refund cap 50) 0.0% 0.0% 0.0%

What to notice: the blocklist is the only row where the train and held-out columns disagree by a meaningful margin (5.2 vs 9.3). Both gates give nearly identical numbers in both columns, which is the table-level signature of a structural fix versus a wording-dependent one.

How to read the attempts-needed table: each row is a different assumed true success rate; the column is how many independent attempts are needed for a 95% chance of seeing at least one success at that rate.

True success rate Attempts for 95% chance of one success
10% 29
5% 59
1% 299
0.1% 2,995

What to notice: the attempts needed roughly scale with the inverse of the rate. Going from 1% to 0.1% (a 10x smaller rate) needs almost exactly 10x more attempts. That is the arithmetic reason a thorough red-team suite run only a few hundred times can confidently rule out a 1% failure mode. But it says very little about anything rarer than that.

How to read the CI-gate table: each row is the same defence configuration as the first table, with the pass/fail rule applied: fail if the interval's lower bound exceeds 2% (or the severity-weighted rate exceeds 2%).

CI rule: fail if the interval's lower bound exceeds 2% (or severity-weighted exceeds 2%) Exit code
No defence 1
Blocklist 1
Gate v1 1
Gate v2 0

What to notice: the blocklist still fails the CI gate despite its large headline improvement over "no defence". A drop from 22% to 7.5% severity-weighted is a real improvement, but it's nowhere near the 2% threshold. That is exactly the point of setting the gate on an absolute limit rather than on "did it get better.".

{{svg:cycle|Generate, attack, measure, fix, re-test with held-out attacks and a regression suite.}} {{svg:results|Attack success rates by defence on train and held-out attack families, with intervals and regression suite sizes.}}