CSE Roadmap

Core tier · about 5 hours · 7 parts

Part 5 of 7 · 2 min read

Speed

In plain words: Which is faster? The measurements use a 256 MiB file that is fully in the page cache, so the disk is not part of the test. The sum of all 8-byte words is computed three ways:

Method Time Rate
read into a 64 KiB buffer, repeated 0.049 to 0.060 s 4.2 to 5.1 GiB/s
mmap, first pass, touching each page 0.049 to 0.057 s 4.4 to 5.1 GiB/s
mmap with MAP_POPULATE: the call itself, then the sum 0.008 to 0.010 s, then 0.040 s

For a sequential pass, the two are close. read copies each block from the cache into the buffer, and mmap pays a page fault for each group of pages. The fault cost is reduced because the kernel maps many neighbouring cached pages at once (the "fault-around" of Level 6 page 7). The flag MAP_POPULATE asks for all the page table entries to be made inside the call, so the later loop meets no faults. This is prefaulting, and it moves the cost to the start. For one pass, the total is similar.

For scattered access the picture changes. The test reads one byte from each of 65,536 pages, in random order:

Method Time per page
pread of one byte (one system call each) 484.5 ns
mmap, first touch 143.5 ns
mmap, second pass (page table entries now exist) 14.5 ns

The mapping wins because there is no system call and no copy for each access, and the second pass costs almost nothing. So mmap suits large files with random access (a database index, an executable's code) and shared read-only data. read suits streams and small files. A program that reads each byte once, in order, gains little from mmap, and read has the advantage that errors come back as return codes and not as signals.

Prefaulting

Prefaulting is the creation of the page table entries of a mapping in advance, as MAP_POPULATE does inside the mmap call, so that later accesses meet no page faults.

Example: mapping the 256 MiB file with MAP_POPULATE took 0.008 to 0.010 s, and the sum loop afterwards ran with no faults.

Check yourself: 1. Why are read and mmap close for a sequential pass? 2. Why is mmap faster for random one-byte reads? 3. What does MAP_POPULATE change?