Part 1 of 4 · 1 min read
Overview
When an agent run goes wrong at step 7 of 12, the final answer is the least informative thing you have. Interviewers ask how you debug and monitor agents in production; the answer is traces with per-step cost, latency and errors, a sampling policy, and links from eval failures (modules 3 and 7) to traces.
About the example, and what is real
Real: the tracer (context manager, span tree, status and error capture, roll-ups, self time), the cost and token aggregation, head and tail sampling, the overhead measurement and the 10 tests in code/mini-tracing. Assumed: the agent is simulated with random durations (model calls 0.6 to 1.4 s with a 3 percent chance of being six times slower; tool calls 0.1 to 0.5 s with a 2 percent timeout chance), token counts, and a price of 3 and 15 dollars per million input and output tokens. Not real: no OpenTelemetry SDK or backend is used; attribute names follow the Gen-AI conventions as I understand them, which are in development. The overhead figure varies by machine.