CSE Roadmap

Core tier · about 4 hours · 13 parts

Part 6 of 13 · 2 min read

What the example found

Filter block rates by attack tier, leak rates by defence and model obedience, and the utility cost of the rule of two.

How to read the filter table: each row is a filter; each tiered column is the share of that tier's 8 attacks it blocked, against 10 benign texts for false positives.

Filter Plain (8) Paraphrase (8) Obfuscated (8) False positives
Naive regex 88% 0% 0% 0 of 10
Normalise, then regex 88% 0% 38% 0 of 10

What to notice: paraphrase sits at 0% for both filters, untouched by the normalisation step that helped obfuscation. Normalisation fixes character-level disguises, not meaning-level ones, which is a structurally different problem no amount of better regex can close.

How to read the leak-rate table: each row is a defence; each column is the model's assumed obedience rate to an injected instruction.

Defence Obeys 30% Obeys 100%
None 28% 92%
Naive filter 20% not run
Hardened filter 16% 50%
Rule-of-two policy 0% 0%

What to notice: every filter-based defence's leak rate rises sharply as obedience rises (16% to 50%), while the rule-of-two policy stays at exactly 0% regardless of obedience. The filters are fighting the model's behaviour and losing ground as that behaviour gets worse, while the policy never depended on the model's behaviour to begin with.

Utility cost: legitimate tasks completed: 100% without the policy, 82% with it.

What to notice: the 18-point utility cost is the direct price of the rule of two. It is paid specifically by the 20% of legitimate tasks that genuinely needed all three properties at once. Those tasks read a web page, consult the private customer table, and email the result. The policy cannot distinguish a legitimate need for all three from an attack that produces the same pattern. So it blocks both alike.

{{svg:trifecta|Untrusted input, private data and external action: allowed pairs and the forbidden triple.}} {{svg:results|Filter block rates by attack tier, leak rates by defence, and the utility cost.}}