Part 4 of 5 · 5 min read
Exercises
Easy
Easy 1: Run and read (20 minutes)
In code/mini-privilege, run pytest and python demo.py. From section 1 of the demo output, list which of the eight attacks setup B stopped and which it did not, and say in one sentence why B did not stop the others.
Done when: you name the two attacks that did harm under B, the two kinds of harm in the output (rows changed, values exposed), and the sentence says that the connection was read-only but reading was not limited.
Hint
B's sql_exec can still SELECT from any table. The harm is in the "values exposed" column, not "rows changed".
Easy 2: Read a refusal (15 minutes)
Take the four refusal messages in section 5 of the demo output. For each, write what a good model should do next, and mark whether the message tells it that.
Done when: you have four rows, and you rewrite at least one message that is not helpful enough.
Hint
the best refusals name the missing thing and give the next move: a hand-off, a different tool, or "do not retry".
Easy 3: Classify these tools (20 minutes)
Give each of these an effect class (read, append, write, money), a permission name, whether it needs approval, and what limit you would put in code: send_email(to, subject, body), read_calendar(day), create_event(title, time, guests), delete_file(path), run_report(name), post_to_channel(channel, text).
Done when: the table has six rows, at least two tools have a limit you can name in code (for example a recipient allowlist or a rate limit), and you mark one choice you are unsure about.
Hint
for send_email the usual limits are an allowlist of recipients or domains, a daily count, and an approval for anything outside a template.
Medium
Medium 1: Add a tool through every layer (60 minutes)
Add reopen_ticket(ticket_id). Give it a scope, a row check, and a decision on approval. Add tests for the scope, the row check and the quota, and make sure lint stays quiet.
Done when: all tests pass, lint(narrow()) is empty, and the tool is denied in a "view" task grant.
Hint
copy close_ticket; the row check and quota come from the gateway if you declare them on the Tool.
Medium 2: A smarter attack: many small refunds (60 minutes)
Add attack A9: refund 5.00 on payment 2 ten times in a row, each under the cap. See how C1 and C2 behave, then close the gap (for example a total refund limit per grant, or a limit per payment per day).
Done when: A9 does harm before your change and no harm after, the other eight attacks and seven tasks give the same results, and a test pins the new behaviour.
Hint
add a refund_total to the grant, increase it on every successful refund, and check it in validate.
Medium 3: Make design B safe (90 minutes)
Keep one SQL tool, but give it a connection that can read only views you define: the current customer's tickets, customers without emails, and a open_tickets_by_plan report. Use the database's authorisation hooks so that any table outside the views is refused.
Done when: attacks A3 and A4 expose nothing, A1 and A2 still fail, the analytics task L6 now passes, and you can say in three sentences what this design costs compared with narrow tools.
Hint
SQLite lets you install an authoriser callback on a connection that sees every table and column a statement touches.
Hard
Hard 1: A real database role (150 minutes)
Repeat design B on a database that has real roles (PostgreSQL, for example). Create a role that can SELECT only from the views of Medium 3, run the SQL tool as that role, and run the eight attacks. (I did not run this, so your numbers are the ground truth.)
Done when: a table shows the result of each attack against the role, with the database's error text where it refused, and you list one thing the role cannot limit (for example the cost of a heavy query).
Hint
grant SELECT on views, not on the base tables, and revoke public access to the schema.
Hard 2: A real model against the poisoned ticket (150 minutes)
Replace the scripted model with a real one, keep the three tool designs A, B and C3, and run the poisoned-ticket task 20 times per design. Record what the model actually asks for. (I did not run this.)
Done when: you report, per design, how many runs tried a harmful call, how many succeeded, and how the model's behaviour differed from the scripted attacker; include one run where the model did something the script never did.
Hint
log the raw tool calls before the gateway, so you see requests that were refused.
Hard 3: Do people read approvals? (150 minutes)
Build a simple approval prompt (command line is fine). Show yourself and at least two other people 40 approval prompts that include three deliberately wrong requests, with the plain "the agent wants to run X. Approve?" message. Then write a better prompt (what will happen, in words; what is unusual) and repeat with new sets. (I did not run this.)
Done when: you report how many wrong requests were approved with each prompt, how long people took, and what you changed in the second prompt; state clearly that the sample is small.
Hint
show the wrong requests at random positions; a person who knows where they are will not behave as in real use.