Part 1 of 6 · 3 min read
Overview
In this module you will
- Explain why a code-running agent needs a sandbox even when the model is well behaved, and name the three ways harm happens: a mistake, an instruction hidden in data, and a trusted helper that is tricked.
- Build the layers of a sandbox from real operating-system mechanisms: limits on time, memory, processes and files; a scrubbed environment; a private working directory; and Linux namespaces for processes, network and files.
- Show, with a table, which layer stops which attack and which attacks nothing short of the last layer stops.
- Explain why filesystem and network isolation are both needed, and demonstrate it on four configurations.
- Say why a filter on the code is not a sandbox, with numbers.
- Write the file API that moves data in and out of the sandbox without handing an attacker a way to read the host.
- Design the tool a model sees so that a timeout, an out-of-memory error and a crash each tell the model what to do next.
- Test a sandbox: a suite of 26 checks, and 29 planted defects to find out which ones it would miss.
- Answer "how would you let an agent run code safely?" in a way that shows you know what a sandbox does not do.
A tool that runs code is the most powerful tool you can give an agent, and the most dangerous one.
Module 1 said: give a tool the least power it needs. A code interpreter is the hard case, because its power is "anything a program can do". The only way to apply least privilege is to draw a wall around the program and decide what is inside. This module builds that wall from real parts and then attacks it.
About the examples, and what is real
Everything runs on this Linux machine, with real operating-system mechanisms: resource limits, process groups, a polling monitor over /proc, and Linux user, process, mount and network namespaces through the unshare and setpriv commands. The attacks are my own short programs, each bounded so that running it bare on a test machine is safe (600 MB of memory, 200 MB of disk, 40 processes). The canary "secrets" are random strings I generated; nothing real is at risk.
Imitated: the "model" is a script (it reacts to phrases in a reply and cannot guess); the honeypot is a small web server on this machine. Missing compared with a production sandbox: a system-call filter (seccomp), control-group limits, and a dedicated unprivileged user. The code runs as "root inside a user namespace with no capabilities", which is a smaller privilege than real root but a larger attack surface than a plain unprivileged user. Not a security product. The results below come from one kernel (6.x, in a container); a kernel without unprivileged user namespaces, macOS or native Windows would behave differently, and the code refuses to run rather than weaken itself (section 4). Hard exercises say "I did not run this, so your numbers are the ground truth".