CSE Roadmap

Core tier · about 3 hours · 9 parts

Part 6 of 9 · 2 min read

Speed of common operations

Rough costs of memory, disk and network operations, and what they say about design.

Think of getting a cup of sugar. Taking it from the shelf next to the stove takes seconds. Walking to the shop across town takes much longer. The shelf is memory. The shop is a faraway data centre.

Latency

Latency is the time one operation takes from start to finish. Small units are used: a millisecond is a thousandth of a second, a microsecond is a millionth, and a nanosecond is a billionth.

Example: a request that takes 150 milliseconds to cross an ocean and return has a latency of 150 ms.

These are rough figures, taken from well-known lists that engineers share (the list is from 2010, and computers are faster now, but the ratios still guide design):

Operation About
Read one value from memory 100 nanoseconds
Read 1 MB in order from memory 250 microseconds
Round trip inside one data centre 500 microseconds
Read 1 MB in order from a solid-state drive 1 millisecond
Seek on a spinning disk 10 milliseconds
Read 1 MB in order from a spinning disk 20 milliseconds
Send a packet across an ocean and back 150 milliseconds

Compare, using whole numbers:

  1. A data centre round trip is 500 microseconds, which is 500,000 nanoseconds. A memory read is 100 nanoseconds. So a round trip is 5,000 times slower than a memory read.
  2. A spinning disk seek is 10 milliseconds, which is 10,000,000 nanoseconds. That is 100,000 times slower than memory.
  3. An ocean round trip is 150 milliseconds. That is 300 data centre round trips.

Design lessons that follow:

  • Memory is fast, and disk is slow. Keep frequently used data in memory.
  • Avoid many small disk seeks. Read data in large blocks if possible.
  • Every network hop costs time. A page that needs 40 internal calls in order is slow, even if each call is quick.
  • Faraway users wait longer. Place data near them.

Check yourself:

  • If a page needs 20 calls in a row inside one data centre, about how much time is that in milliseconds?
  • Why is a spinning disk seek much worse than a memory read?