CSE Roadmap

Core tier · about 4 hours · 13 parts

Part 11 of 13 · 1 min read

Quick notes

  • Prompt injection is untrusted text being treated as instructions. The model has one input channel, so instructions and data arrive mixed, and the model is trained to follow instructions.
  • Indirect injection is the main threat for agents. The attack sits in a web page, an email or a tool result the agent reads, and the user never sees it. It is LLM01 in the OWASP Top 10 for LLM Applications.
  • The lethal trifecta: private data, untrusted content and the ability to communicate externally. An agent should not have all three at once (the same idea as the Agents Rule of Two).
  • Filters help and then fail. In the example a regex caught most plain attacks, none of the paraphrased ones, and only some obfuscated ones even after normalising the text.
  • Architecture holds when the model fails. A rule-of-two policy leaked nothing even when the model obeyed every injected instruction, at the price of blocking some legitimate tasks. The quarantine pattern lets an unprivileged model read untrusted text and return only validated fields.