Part 2 of 13 · 2 min read
An agent holding more power than its task needs is a bigger bet on the model never slipping
The amount of damage a compromised agent can do is a number set entirely by what you handed it, not by how it was tricked.
Think of a hotel key card that opens only your room, for your stay. If it is stolen, the thief gets one room, not the building. The card's narrowness, not any trust in the thief, is what limits the damage.
excessive agency is an agent holding functionality, permissions or autonomy beyond what its task needs, and least privilege is the opposite discipline. Grant only the access required for the task, for the time required -- which matters because it is the one control that still works on the day the model is fooled into obeying an attacker, since it limits what a successfully-fooled agent can even ask for.
A worked example. Simulating a fully compromised support agent making 20 attempts from a 28-action catalog, the worst-case damage it can do is 27,187 assumed units under one broad service-account credential, 4,530 units under tool-scoped grants (read customers, refunds capped at 500, internal email only), and just 53 units under per-task grants (one customer, one capped refund, three reads, an expiry). The agent is equally "compromised" in all three cases -- the only thing that changed is what it was handed in the first place.
The key lines. From code/mini-privilege/privilege.py:
# from code/mini-privilege/privilege.py
def design(name, now_expiry=100.0):
if name == "god":
return [Grant("*", frozenset({"read", "refund", "delete", "write", "send", "run", "exec", "create"}))]
if name == "tool_scoped":
return [Grant("customers/*", frozenset({"read"})), Grant("orders/*", frozenset({"read", "refund"}), max_amount=500.0),
Grant("email/internal", frozenset({"send"}))]
Read it as: the "god" design is a single grant matching every resource ("*") with every action in the catalog. This one line is excessive agency made literal. The agent does not do anything differently when compromised; the credential simply lets a compromised agent reach every single thing in the 28-action catalog. The scoped version physically cannot express most of those actions at all.
Excessive agency, least privilege
Excessive agency is an agent holding functionality, permissions or autonomy beyond what its task needs. Least privilege is granting only the access required for the task, for the time required.
Example: the jump from 27,187 to 53 worst-case damage units, for the identical compromised agent, is excessive agency and least privilege shown as the same number measured twice -- once with the power present, once with it removed.