CSE Roadmap

Common tier · about 4 hours · 7 parts

Part 4 of 7 · 2 min read

Transparent huge pages

In plain words: The second way needs no set-up. Transparent huge pages (THP) let the kernel use a huge page whenever a region is suitable: an anonymous region that is aligned to 2 MiB, wholly inside the mapping, and touched for the first time. The page fault handler then allocates one 2 MiB frame, zeroes it, and fills one page table entry, instead of 512 faults. The program is not changed.

The system setting is in /sys/kernel/mm/transparent_hugepage/enabled, with three values: always (use huge pages wherever possible), madvise (only for regions that ask with madvise(addr, len, MADV_HUGEPAGE)) and never. The test machine reads always [madvise] never, so the brackets mark madvise. The test below maps 64 MiB, aligned to 2 MiB. It touches one byte in each 4 KiB page:

Region Faults Time (three runs) AnonHugePages RSS
MADV_NOHUGEPAGE 16,387 0.026 to 0.034 s 0 kB 67,184 kB
MADV_HUGEPAGE 35 0.010 to 0.021 s 65,536 kB 67,180 to 67,184 kB

The number of faults fell from 16,387 to 35. Thirty-two of them were for the 32 huge pages of 2 MiB, and a few belonged to the program itself. The first touch ran about 1.3 to 3 times as fast, with a wide spread between runs. The kernel counted the huge pages in the field AnonHugePages (65,536 kB, the whole region). The counters thp_fault_alloc and thp_fault_fallback in /proc/vmstat show how many huge pages were given. They also show how often the kernel found no free aligned 2 MiB frame and fell back to base pages (0 here). When memory is fragmented, a contiguous 2 MiB range may not exist, and the kernel can run compaction, moving pages together to make one, which costs time.

Transparent huge pages

Transparent huge pages (THP) are a Linux feature in which the kernel backs suitable anonymous regions with 2 MiB pages, with no change to the program. It does so everywhere, or only where the program asks with madvise(MADV_HUGEPAGE).

Example: with the setting madvise, a 64 MiB region marked MADV_HUGEPAGE was touched with 35 faults and not 16,387.

Read it as

In /proc/PID/smaps_rollup, the line AnonHugePages shows how much of the process is held in huge pages now. A value of 0 means the process uses none.

Check yourself: 1. What are the three settings of THP? 2. By how much did the number of faults fall in the test, and why? 3. What does the kernel do if no free aligned 2 MiB range exists?