CSE Roadmap

Core tier · about 4 hours · 13 parts

Part 6 of 13 · 2 min read

What the example found

Argument-validator comparison, output-guard coverage against attacks and evasions, the correlation sweep, and the fail-open/fail-closed trade-off.

How to read the argument table: each row is a validator design against the same 14 hostile or malformed argument sets, with all 4 legitimate calls accepted by both.

Check Legitimate allowed Bad inputs accepted
Keys present only 4 of 4 93% (13 of 14)
Strict schema 4 of 4 0%

What to notice: both validators pass every legitimate call, so the strict schema's near-total rejection of bad inputs comes at zero cost to legitimate usage in this test. There's no trade-off visible here, only a gap in how much the naive check was actually checking.

How to read the output-guard table: each row is a set of outputs; the column is the share flagged.

Output guard Flagged
10 plain exfiltration outputs 100%
5 evasions 0%
10 benign outputs 0%

What to notice: the guard's behavior on evasions (0%) and on benign outputs (0%) looks identical in this table, but they mean opposite things. One is the guard correctly staying silent, the other is the guard being completely blind. The number alone can't tell you which, which is exactly why evasions need their own labeled test set rather than being folded into "false positive rate.".

How to read the correlation table: each row raises the assumed correlation between what the three layers miss, with each layer's own miss rate fixed at 20%.

Three layers, each missing 20% Attacks through all three
Independent 0.8%
Correlation 0.5 5.0%
Correlation 0.9 12.8%
Same blind spot 20.0%

What to notice: the combined miss rate at full correlation (20.0%) exactly equals a single layer's own miss rate. Which is the mathematical signature of "these aren't three layers, they're one layer wearing three coats," and a useful sanity check to run on any claimed defence-in-depth setup.

How to read the fail-mode table: each row is a failure policy under a guard that itself errors on 5% of calls.

Guard errors on 5% of calls Attacks passed Good outputs blocked
Fail closed 0.0% 4.2%
Fail open 5.2% 0.0%

What to notice: fail open's attack-pass-through rate (5.2%) is almost exactly the guard's own error rate (5%). Every time the guard breaks, an attack that would otherwise be screened simply walks through unchecked. That is the concrete mechanism behind "fail open on a security check is equivalent to not having the check during every outage.".