Part 2 of 13 · 2 min read
A trace of spans turns "it gave the wrong refund" into something checkable
A vague complaint about an agent is unanswerable until it is broken into the individual steps the agent actually took.
Think of a black-box flight recorder: after a crash, investigators don't start from theories, they start from the recorded sequence of instrument readings leading up to it.
In plain words: a trace is the record of one request, made of spans — one per guard, retrieval, planner call, tool call or answer — each with its own duration, status, arguments and sizes, and a swallowed error is the specific, common failure pattern where a tool span's status is "error" but the agent's final answer carries on as if nothing went wrong.
A worked example. In the synthetic support-agent traces, a swallowed-error fault is detected simply by checking whether the last tool span in a trace has status == "error" and the final answer was never hedged to reflect that failure. Across 6,000 synthetic traces, this single structural check caught all 169 true swallowed-error cases with zero false positives — because the pattern it's looking for ("a tool failed, and nothing downstream acknowledges it") is written directly into the structure of the spans.
The key lines. From code/mini-debug/detect.py:
# from code/mini-debug/detect.py
def tool_spans(t):
return [s for s in t.spans if s.kind == "tool"]
def swallowed_error(t):
ts = tool_spans(t)
return bool(ts) and ts[-1].status == "error" and not t.hedged
Read it as: the check only needs two fields on two objects: ts[-1].status and t.hedged. This is the entire value of structured tracing. A bug that would take real effort to spot by reading prose output becomes a two-line check. That happens once the agent's steps are recorded as spans with a status field.
Trace and span, swallowed error
A trace is the record of one request; a span is one step in it (a guard, retrieval, planner call, tool call or answer) with its duration, status, arguments and sizes. A swallowed error is a tool failure that the agent continues past as if it had succeeded.
Example: "the refund tool span had status error, there was no later tool call, and the answer did not hedge" is a swallowed error stated entirely in terms of spans — which is exactly the vocabulary an interviewer wants to hear you reach for before any talk of causes.