CSE Roadmap

Common tier · about 4 hours · 12 parts

Part 10 of 12 · 1 min read

Quick notes

  • Red-teaming is structured, repeatable attack practice on your own system, run as a lifecycle: implement, attack, evaluate, mitigate, repeat.
  • Build a generator, not a list. Crossing goals, carriers, phrasing families and obfuscations shows which category fails, not just that something did.
  • Report an attack success rate with a confidence interval, and weight it by severity: a high rate on low-harm goals is a different risk from a low rate on data exfiltration.
  • Fixes that learn the attacks you saw overfit them. A blocklist did well on training attacks and worse on held-out phrasing families; a structural gate on the action did not care about phrasing.
  • Zero is not safe. A zero rate says no known attack worked. Keep every successful attack as a regression test and fail the build only when the interval's lower bound exceeds the allowed rate.