CSE Roadmap

Specialist tier · about 3 hours · 12 parts

Part 9 of 12 · 2 min read

Accountability and escalation: common mistakes

Common mistakes, and what the example does not do.

Mistakes that make accountability hollow

  • Logging for debugging only, with no schema, owner or retention.
  • Logs that the agent can write to or delete.
  • A hash chain with no anchor, assumed to be immutable.
  • No severity definitions until the incident.
  • One on-call person and repeated pages.
  • Escalation judged by the model alone.
  • Kill switch that stops the chat but not the tools.
  • A breaker threshold chosen without baseline traffic.
  • No owner for approvals or incident response.

What the example does not do

About this example What the example does not do
  • In-memory log, one writer, no concurrency, no rotation, no storage guarantees.
  • The anchor is a Python tuple; a real anchor needs a separate trusted system.
  • Simulated people and attacks: availability, rates and flagged fractions are assumptions; real paging has sleep schedules, shared channels and alert fatigue.
  • Severity weights are arbitrary, not calibrated against an organisation's risk appetite.
  • The breaker counts events in a sliding window, with no per-user or per-tool scoping.
  • No legal, regulatory or contractual analysis of responsibility.
  • Optional real experiment: run a game day on a real agent: trigger a synthetic SEV2, time the acknowledgement, use the kill switch and reconstruct the incident from the audit log alone.

Check yourself

Who is accountable when an agent causes harm?

Show answer

A named human or team in the organisation. The agent cannot be accountable, and you need logs to show what happened.

For the curious A hash chain is not enough

Of five tampering attacks, a plain log detected 0. A hash chain detected edits, deletion and reordering but not tail truncation or a full rewrite by someone who recomputes the hashes. Re-paging the same person every 15 minutes never reached a human in 10 percent of incidents at 90 percent availability, and 30 percent at 70 percent. The attack, availability and traffic here are simulated.

What you will use daily at work

Practice What it looks like
Log nine fields for each consequential action Who, action, argument hash, decision, approver, model and prompt version, trace id, time.
Write the severity rule before an incident Impact, reach and reversibility to SEV1 to SEV4.
Escalate to a rota, not a person Re-page a different person if nobody answers.
Add a circuit breaker with a named human reset It halts the agent after N suspicious events.