Part 8 of 12 · 2 min read
Answering agent-interface questions in an interview
How would you show an agent's progress without overwhelming the user?
"In layers: a one-line status, a separate 'needs you' area for approvals and unrecovered errors, and a collapsed timeline that groups repeated steps and shows details on demand. Add a list of changes with undo, pause and cancel. The default view is built from the event stream (Level 11 module 4) and critical events are exempt from collapsing. In the example's simulation a run of 82 events reduced to about 26 grouped items or 1.4 in the summary view without hiding any unresolved approval or error." Follow-up: "What about power users?" An expanded view with full traces, tool inputs and outputs, and token costs, one click away.
How do you design approvals so people do not just click yes?
"Reserve prompts for what matters: auto-run low-risk reversible actions with undo, batch medium ones, and show a preview with explicit confirmation for high-risk or irreversible ones, with a default of deny on timeout. Under an assumed fatigue model, asking about everything meant only about half of the risky actions were actually read." Follow-up: "How do you know it works?" Plant a risky action in testing. Measure whether users catch it.
How should an agent communicate uncertainty?
"Only with calibrated confidence, in plain wording tied to sources, and by asking a focused question when two readings are close. I choose the ask margin from the cost of a wrong action. In the example's simulation, raising it from 0.1 to 0.3 cut wrong actions from 4.9 to 0.3 percent but raised questions from 22 to 65 percent." Follow-up: "What if the model's confidence is not calibrated?" Calibrate it on labelled data, or show evidence instead of confidence words.
What does undo look like for an agent that sends emails and edits files?
"File edits get inverse operations or versioning; emails get a delayed-send window; anything truly irreversible is labelled and needs confirmation. An action log shows what changed and lets the user revert." Follow-up: "What about actions in third-party systems?" Use compensating actions where they exist (cancel, refund) and treat the rest as irreversible.