Part 8 of 12 · 2 min read
Accountability and escalation: interview questions
Who is accountable when an agent causes harm?
"The organisation and, inside it, a named owner. Technically I make that answerable: an audit log of who or what acted, with which arguments, who approved, which model and prompt version, and a trace id; a RACI for approvals and on-call; and escalation paths. The legal question depends on contracts and law, so I involve counsel; I treat it as outside engineering." Follow-up: "What do you log?" The nine fields in my record, with personal data redacted, in an append-only store.
How do you make an audit log trustworthy?
"Chain the entries by hash so edits, deletions and reordering are visible, keep an HMAC key or the anchor outside the log's storage, and check regularly. In the example's test a plain log detected none of five attacks, a hash chain missed truncation and a rewrite, and an anchored chain caught all five." Follow-up: "Is that immutable?" No: it is tamper-evident. For prevention use write-once storage and tight access control.
How do you design escalation?
"Rule-based triggers for high-stakes cases, such as value thresholds or irreversible actions, severity levels defined in advance, and paging that moves to different people after a timeout, because unavailability is correlated within a person. In the example's simulation re-paging one person never reached a human in 30 percent of incidents at 70 percent availability, escalating cut that to 2.5 percent." Follow-up: "How do you test it?" Game days: page the path at random times and measure time to acknowledge.
How do you stop an agent quickly?
"A circuit breaker on suspicious events plus a manual kill switch that revokes tool credentials, with a named human to reset. I choose the threshold from baselines: in the example's simulation limit 3 cut harm from 68.5 to 2.3 units but tripped about three times in 8 hours on normal traffic, limit 10 almost never tripped." Follow-up: "What about in-flight actions?" Make actions idempotent and reversible where possible (Level 8 module 6) and queue irreversible ones behind approval.