Part 2 of 12 · 2 min read
Why identity must come from the session, not the model
Trusting a model-supplied tenant id lets an injected instruction ask for someone else's data.
Think of a hotel key card that checks whatever room number a guest types into a keypad, instead of checking which key card they actually hold. Anyone who types a different number walks into a different room.
In plain words: if a tool trusts a tenant id that the model supplies as an argument, a prompt injection can make the model ask for another tenant's data. The tool will then hand it over.
A worked example. With 1,000 injected attempts and a model that obeys the injected instruction 30 percent of the time, a tool that trusts the model's tenant_id argument leaks data 306 times. The same tool, changed to ignore that argument and use the tenant bound to the authenticated session instead, leaks 0 times — even when the model obeys every single injection.
The key lines. From code/mini-tenancy/tenancy.py:
# from code/mini-tenancy/tenancy.py
def tool_arg_scoped(self, ctx, tenant_arg): return self.data.get(tenant_arg, []) # trusts the model's argument: confused deputy
def tool_session_scoped(self, ctx, tenant_arg=None): return self.data.get(ctx.tenant, []) # ignores any tenant the model supplies
Read it as: one version looks up data by whatever the model handed it. The other looks up data by the identity the platform already verified, and the second argument is simply thrown away. The fix costs nothing at call time. It is just a decision about which variable to trust.
Confused deputy
A component with real authority (the tool) is tricked by an untrusted input (a model-chosen argument) into using that authority on behalf of someone who shouldn't have it.
Example: the 306 leaks above, each one the tool acting on an argument instead of on the session that actually authenticated.