Part 9 of 12 · 2 min read
Mistakes that make traces useless
Common mistakes, and what the example does not do.
Mistakes that make traces useless
- Logs instead of traces, so steps cannot be tied to a run.
- Head-sampling at 1 percent and then missing every rare failure.
- No token or cost attributes.
- Capturing full prompts with personal data and no retention.
- Broken context propagation, so tool calls appear as separate traces.
- Hard-coding attribute names from a draft convention without a version note.
- Tracing only the model call, not tools and retrieval.
- No link between eval results and traces.
What the example does not do
About this example What the example does not do
- No OpenTelemetry SDK, exporter, collector or backend; the tracer is minimal and synchronous.
- A simulated agent with simple durations; real runs have retries, parallel calls and streaming.
- Self time assumes sequential children; parallel spans need the critical-path calculation.
- The tail rule is evaluated offline on complete traces; a real collector must buffer.
- The convention status comes from a secondary source; the specification was not read.
- Optional real experiment: instrument a real agent with the OpenTelemetry SDK, export to a local backend, and compare your measured per-span overhead and attribute names with this module.
Check yourself
A run was slow. How do you find out whether the model or a tool was the problem?
Show answer
Read the trace: the span tree shows start, end and parent for each model and tool call. In the simulated run, model calls were 2.94 s of 3.28 s.
For the curious Sampling decides what you can debug later
Over 20,000 simulated runs, 3.9 percent had a tool error. Random head sampling at 1 percent kept only 1 percent of those error traces; deciding after the run (a tail rule) can keep the errors you need. The OpenTelemetry Gen-AI conventions were still marked Development in the secondary source read, so record which version of the names you used.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| One trace per request | Spans for each model call, tool call, retrieval and handoff. |
| Record tokens, model, tool and error type | Content only on purpose, redacted, with a short retention. |
| Keep error traces | Sample after the run, not only at the start. |
| Store the trace id with each eval result | A red build then points to a run. |