Part 6 of 9 · 2 min read
Speed of common operations
Rough costs of memory, disk and network operations, and what they say about design.
Think of getting a cup of sugar. Taking it from the shelf next to the stove takes seconds. Walking to the shop across town takes much longer. The shelf is memory. The shop is a faraway data centre.
Latency
Latency is the time one operation takes from start to finish. Small units are used: a millisecond is a thousandth of a second, a microsecond is a millionth, and a nanosecond is a billionth.
Example: a request that takes 150 milliseconds to cross an ocean and return has a latency of 150 ms.
These are rough figures, taken from well-known lists that engineers share (the list is from 2010, and computers are faster now, but the ratios still guide design):
| Operation | About |
|---|---|
| Read one value from memory | 100 nanoseconds |
| Read 1 MB in order from memory | 250 microseconds |
| Round trip inside one data centre | 500 microseconds |
| Read 1 MB in order from a solid-state drive | 1 millisecond |
| Seek on a spinning disk | 10 milliseconds |
| Read 1 MB in order from a spinning disk | 20 milliseconds |
| Send a packet across an ocean and back | 150 milliseconds |
Compare, using whole numbers:
- A data centre round trip is 500 microseconds, which is 500,000 nanoseconds. A memory read is 100 nanoseconds. So a round trip is 5,000 times slower than a memory read.
- A spinning disk seek is 10 milliseconds, which is 10,000,000 nanoseconds. That is 100,000 times slower than memory.
- An ocean round trip is 150 milliseconds. That is 300 data centre round trips.
Design lessons that follow:
- Memory is fast, and disk is slow. Keep frequently used data in memory.
- Avoid many small disk seeks. Read data in large blocks if possible.
- Every network hop costs time. A page that needs 40 internal calls in order is slow, even if each call is quick.
- Faraway users wait longer. Place data near them.
Check yourself:
- If a page needs 20 calls in a row inside one data centre, about how much time is that in milliseconds?
- Why is a spinning disk seek much worse than a memory read?