CSE Roadmap

Common tier · about 4 hours · 12 parts

Part 7 of 12 · 1 min read

How to build it so it holds up

  • Generate attacks from a grid of goals, carriers, phrasings and obfuscations, and add new cells when threat modelling (module 7) finds new goals.
  • Run each attack many times (the model is stochastic) and report ASR with an interval.
  • Split attacks into design and held-out sets before fixing, and report both.
  • Prefer structural fixes (module 2 least privilege, module 3 sandbox, module 5 approvals) and measure them like any other.
  • Keep a regression suite of every successful attack and gate CI on it.
  • Weight by severity and report which goals remain.
  • State what zero means (attempts and coverage), not "secure".
  • Use human red-teamers and tools for creativity (automated generators cover known patterns); rotate them and share findings.
  • Re-run on every change to model, prompt, tool or data, and on a schedule.
  • Disclose and track findings like bugs, with owners, severities and fix verification.
  • Test responsibly: only systems you own or have written permission to test; no real personal data in attack sets.