CSE Roadmap

Core tier · about 3 hours · 8 parts

Part 6 of 8 · 2 min read

Traces

The path of one request across many services.

A single log event covers one service. A user request may cross five. Tracing records the whole life of a request as it moves through the system. A trace is a list of causally related spans. A span is a stretch of time that stands for one step of work, and it carries key and value pairs. UDS shows the execution flow of a request as spans on a time line.

How it works: 1. When a request starts, it is given a unique trace id. 2. The id travels at every hand-off: between threads, and across the network from caller to callee, for example in an HTTP header. 3. Each stage is a span, an event that includes the trace id. 4. When a span ends, it is sent to a collector that assembles the spans of one trace. UDS names Open Zipkin and AWS X-Ray.

A made-up trace for a 480 ms page load: gateway span 480 ms; inside it the cart service 60 ms and the pricing service 400 ms; inside pricing a database call of 350 ms. The time line shows at once that the database call causes most of the delay.

UDS says traces let you: - debug problems for specific requests, such as a support ticket; - debug rare problems affecting a tiny fraction of requests; - debug problems affecting many requests, such as slow answers from a subset of instances; - find bottlenecks along the path; - see which clients call which downstream services and in what share (resource attribution), useful for rate limits or billing.

Hard to add later. Tracing is difficult to retrofit, since every component on the path must pass the context along, including your frameworks, libraries and third-party services. UDS notes that the service mesh pattern (Level 7) can help.

Trace and span

A trace is the complete record of one request across services. A span is one timed step inside it.

Example: trace t-77 has a gateway span of 480 ms that contains a pricing span of 400 ms.

Check yourself: 1. What carries a trace across a network call? 2. What is the difference between a span and a trace? 3. Why is tracing hard to retrofit?