CSE Roadmap

Core tier · about 2 hours · 6 parts

Part 3 of 6 · 2 min read

Throughput, response time, latency

Think of throughput like how many cars a toll booth processes per hour. Response time is how long one specific driver waits from arriving to driving away, including time stuck in the queue before reaching the booth.

In plain words: Throughput is the number of records or requests a system processes per unit of time, the natural measure for a batch job that just needs to finish a large amount of work. Response time is what a client actually experiences: the time from sending a request to receiving a reply, the natural measure for an online, interactive system. Latency and response time are often used as if they mean the same thing, but they do not. Latency is specifically the time a request spends waiting to be handled, before any actual work on it begins; response time is the full client-visible total, the service time plus any network delay plus any queueing delay, including that latency.

A worked example. A checkout request to the bookstore's database sits in a queue for 40 ms behind other requests, then takes 10 ms to actually execute, then 5 ms to travel back over the network. Its latency is 40 ms; its full response time is 55 ms.

The key lines. No new SQL; these are measured durations, not query syntax.

queueing delay (latency):  40 ms
service time:              10 ms
network delay:              5 ms
total response time:       55 ms

Throughput, response time, latency

Throughput is the number of operations a system completes per unit of time. Response time is the total client-visible duration from sending a request to receiving its reply. Latency is specifically the portion of that duration spent waiting before the request is actually handled.

Example: in the checkout request above, the 40 ms queueing delay is latency, while the full 55 ms is response time.