CSE Roadmap

Core tier · about 5 hours · 7 parts

Part 4 of 7 · 3 min read

Minor and major faults

In plain words: Linux counts the faults of each process and reports two numbers. A minor fault is one that the kernel can fix without waiting for the disk: the page is already in memory (for example in the file cache, or a page of zeros is needed), and only the page table must change. A major fault needs a disk read. The program below allocates 64 MiB with mmap, touches one byte in each page, and prints the counts from getrusage:

#define _GNU_SOURCE
#include <stdio.h>
#include <sys/mman.h>
#include <sys/resource.h>
int main(void){
  size_t ps=4096, n=16384;                      /* 64 MiB */
  char *m=mmap(0,n*ps,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0);
  if(m==MAP_FAILED){ perror("mmap"); return 1; }
  madvise(m,n*ps,MADV_NOHUGEPAGE);              /* keep 4 KiB pages */
  struct rusage a,b;
  getrusage(RUSAGE_SELF,&a);
  for(size_t i=0;i<n;i++) m[i*ps]=1;            /* first touch of every page */
  getrusage(RUSAGE_SELF,&b);
  printf("pages %zu, minor faults %ld, major faults %ld\n",
         n, b.ru_minflt-a.ru_minflt, b.ru_majflt-a.ru_majflt);
  return 0; }

Output on the test machine: pages 16384, minor faults 16384, major faults 0. The mmap call itself used no memory: it only created a region. Each of the 16,384 first touches caused one minor fault. The madvise call turns off huge pages (2 MiB pages, a feature of the processor that lets one entry cover many ordinary pages); with huge pages on, the same loop would give far fewer faults.

The same program, with a second timed pass, shows the cost:

Measurement Result
First touch of 16,384 anonymous pages 0.029 s, about 1,770 ns per page, 16,387 minor faults
Second pass over the same pages about 19 to 26 ns per page, no faults
File mapping of 64 MiB, first pass, file not in the cache 1 major fault and 516 minor faults, 0.055 s
File mapping again, file in the cache 516 minor faults, no major fault, about 52 to 62 ns per page

Three facts stand out. First, a minor fault costs roughly 1.8 microseconds here, about 70 to 90 times a normal access to the same page. Second, the file mapping needed 516 faults for 16,384 pages: the kernel maps many neighbouring pages that are already in the cache at each fault (on this machine about 32 pages per fault, and the number depends on the kernel and the file system). Third, the single major fault in the table does not show what a disk costs. The storage of the virtual machine is very fast, and a spinning disk needs milliseconds, as the next section uses.

Major page fault

A major page fault is a page fault whose handling must wait for a read from storage, while a minor page fault is one that the kernel fixes with no storage read, such as supplying a zeroed page or mapping a page already in memory.

Example: the first touch of a freshly allocated page is minor, and the first touch of a page of a program file that is not in the cache is major.

Read it as

On Linux, /usr/bin/time -v ./prog prints "Major (requiring I/O) page faults" and "Minor (reclaiming a frame) page faults" for a run.

Check yourself: 1. Why did 16,384 touches cause 16,384 minor faults and not major faults? 2. Why did the second pass cause no faults? 3. Why does the file mapping show fewer faults than pages?