Part 4 of 6 · 2 min read
Percentiles not averages
Think of a sorted line of runners finishing a race. The median is whoever crosses at the halfway point in that line. The 99th percentile is near the very back, just before the slowest one percent.
In plain words: A single request's response time varies from one try to the next, so a system handling many requests is better described by a distribution of response times than by one number. The average, the arithmetic mean, is commonly reported but is a poor measure of what a typical user actually experiences, since a handful of very slow outliers can pull the mean upward without most users ever feeling it. Percentiles are the better tool. The median, or 50th percentile (p50), is the point where half of requests are faster and half are slower. Higher percentiles, such as the 95th, 99th, or 99.9th, show how bad the slow outliers actually are: a p99 of 1.5 seconds means 99 out of 100 requests finish faster than that, and 1 out of 100 takes 1.5 seconds or longer.
A worked example. The bookstore's checkout endpoint has a median response time of 120 ms, but its p99 is 2.1 seconds. Most customers barely notice any delay, but roughly 1 in 100 checkouts feels sluggish, exactly the kind of gap an average alone would hide.
The key lines. No new SQL; percentiles are computed over measured response times, not queried from a table in the usual sense.
sorted response times (ms): [80, 95, 100, 110, 120, 140, 180, 900, 2100]
median (p50): 120 ms
p99 (roughly): 2100 ms
Percentile, median (p50)
A percentile is the response-time threshold below which a given percentage of requests fall, once all response times are sorted from fastest to slowest. The median, or p50, is the 50th percentile, the halfway point of that sorted list.
Example: the checkout endpoint's p99 of 2.1 seconds means 99 out of every 100 requests finish faster than that.