CSE Roadmap

Core tier · about 3 hours · 7 parts

Part 2 of 7 · 2 min read

What to measure

Think of a toll road. You can count how many cars pass each second, how many tonnes of goods they carry, how long a car takes to cross, and how many cars are on the road at once. The four numbers describe the same road in four ways. The analogy stops in one place: cars on a road never wait inside the toll booth, while requests wait in the queue of the device.

In plain words: IOPS (I/O operations per second) is the number of requests that finish in each second. Throughput is the bytes that move per second, which is IOPS times the request size. Latency is the time from the start of one request to its end. Queue depth is the number of requests that are in flight at one moment. Utilisation is the share of time in which the device had at least one request. The numbers are linked by Little's law: the average number of requests in flight equals the rate of requests times the average time that each takes.

A worked example. A disk finishes 20,000 requests per second, and each takes 0.5 ms on average. The average number in flight is 20,000 x 0.0005 = 10. If each request is 4 KiB, the throughput is 20,000 x 4,096 = 81.9 MB/s. If the same disk finished 1,000 requests of 1 MiB each per second, the IOPS would be only 1,000 but the throughput would be 1,048.6 MB/s. A single figure alone does not describe a disk.

The key lines. Read IOPS for small random loads, throughput for large sequential loads, and latency for loads in which a person or a program waits for each answer. Page 1 explained why the first group is much slower on a spinning disk.

IOPS

IOPS is the number of I/O operations that a device completes in one second. It is a count of requests and does not tell their size. Throughput is IOPS multiplied by the average request size.

Example: 20,000 IOPS of 4 KiB requests is a throughput of about 81.9 MB/s.

Little's law

Little's law says that, in a stable system, the average number of items in the system equals the average arrival rate times the average time that an item spends in the system. For a disk, the average queue depth equals the IOPS times the average latency.

Example: 20,000 requests per second at 0.5 ms each give an average of 10 requests in flight.

Check yourself: 1. A disk does 5,000 IOPS of 8 KiB requests. What is the throughput? 2. A disk has an average of 4 requests in flight and a latency of 2 ms. What is its IOPS? 3. Why does a single figure not describe a disk?