Part 7 of 13 · 5 min read
A filter is not a sandbox
Why a filter on the code is not a sandbox, and the file API.
Reading the code is not the same as limiting what it can do
A bouncer with a list of banned words stops the obvious phrasing; a clever visitor just says it a different way. A filter reads what the code says; a sandbox limits what the code can do, no matter how it is written.
In plain words: checking the text of a program before it runs can only catch phrasings someone thought to ban, while a real sandbox limits what the running program can do regardless of how it was written.
The tempting shortcut is to look at the code before running it and refuse anything dangerous. The example includes that filter (it parses the code and flags imports of os, subprocess, socket and others, and uses of open, eval and exec) and measures it on the same programs.
| Programs | Flagged | Not flagged |
|---|---|---|
| 13 harmful | 9 | 4: loop, sleep, memory, flood (nothing in the code looks dangerous; the danger is in how much) |
| 3 harmful programs, spelled differently | 1 | 2: a module imported through importlib with a name built from pieces, and a function fetched with getattr |
| 4 ordinary programs | 3 (false alarms) | 1: anything that opens a file, runs a subprocess or fetches a page is refused |
Static filter, resource attack
A static filter inspects source text before running it and looks for named patterns. It cannot see resource attacks (a loop, a large allocation), because nothing in the text of while True: pass names a resource.
Example: importlib.import_module("os"[::-1][::-1]) does whatever a plain import os would, and the filter's pattern for the literal word os never matches it.
A filter catches the obvious spelling, misses resource attacks entirely, can be walked around by anyone who writes the same thing differently, and blocks ordinary work. What it is good for is a cheap, early hint to the model ("this sandbox has no network, so do not import socket"). Keep it as a courtesy and let the sandbox be the defence.
The file API: code that runs outside the wall
Data has to get in and out. A tool that accepts "put this file" and "give me that file" runs on the host, with the host's privileges, and the name comes from code running inside. That is a recipe for a confused deputy: a privileged helper that can be steered into using its privilege for someone who has none.
The rules the example implements are each one check inside _resolve:
# from code/mini-sandbox/sandbox.py
def _resolve(self, name: str) -> str:
if not isinstance(name, str) or not name or "\x00" in name:
raise FileError("a file name must be a non-empty string without NUL")
if os.path.isabs(name) and "abs_path_ok" not in self.bugs:
raise FileError("absolute paths are not allowed; use a name relative to the working directory")
parts = name.split("/")
if ".." in parts and "traversal_ok" not in self.bugs:
raise FileError("'..' is not allowed in a file name")
path = os.path.join(self.dir, name)
if "symlink_followed" not in self.bugs:
real_root = os.path.realpath(self.dir)
real = os.path.realpath(path)
if os.path.commonpath([real_root, real]) != real_root:
raise FileError("that name leads outside the working directory (through a link)")
return path
Read it as: the last check resolves the name all the way through any symbolic link and compares it against the resolved root. So a file named report.txt that is secretly a link pointing at the key's real path is caught here. Neither the "no absolute path" check nor the "no .." check above it would ever see a problem with that name.
Confused deputy, symbolic link
A confused deputy is a privileged piece of code tricked into misusing its own privilege on behalf of something that has none. A symbolic link is a file that is really just a pointer to another path.
Example: the sandboxed program creates a link named report.txt pointing at the hidden key's real location; if the host's file API followed it, reading report.txt would read the key.
In the experiment: with the check, get refuses ("that name leads outside the working directory"). With the check removed, the secret came back to the host. Only ordinary files are read. Links are never followed, even to open the file. A link to a harmless file inside the directory is still refused, which is the choice made here. That also closes the gap between checking a name and opening it. A named pipe would make read wait forever, so the type is checked on the open file. There are also size limits both ways, a file count, a disk quota, and a run count.
Two honest findings from testing this layer. First, the "no .." check and the "no absolute path" check are redundant with the link check, which also refuses those names. Remove either one alone and nothing in the test suite fails. They are equivalent mutants (part 11). Removing all three together fails three checks. Second, a check that looks like it covers one rule may be covered by another. A suite cannot tell which guard is doing the work unless you take them away one at a time.