Part 7 of 7 · 3 min read
Exercises
Easy
Easy 1: Entries and sizes (20 minutes)
How many TLB entries cover 512 MiB with 4 KiB pages, and with 2 MiB pages?
Done when: 131,072 and 256.
Hint
Divide 512 MiB by 4 KiB, and by 2 MiB.
Easy 2: Reach of a TLB (20 minutes)
A TLB has 1,536 entries (made up). Find its reach with 4 KiB and with 2 MiB pages.
Done when: 6 MiB and 3,072 MiB (3 GiB).
Hint
Multiply the entries by the page size.
Easy 3: Read the setting (20 minutes)
Run cat /sys/kernel/mm/transparent_hugepage/enabled on your machine. Which mode is chosen, and what does it mean for a program that never calls madvise?
Done when: a mode name read from the brackets, and a sentence that says whether the program gets huge pages (it does only under always).
Hint
The word in square brackets is the active mode.
Medium
Medium 1: Count the faults (45 minutes)
Write the program of section 3. Map 64 MiB aligned to 2 MiB and mark it MADV_HUGEPAGE or MADV_NOHUGEPAGE. Touch one byte in each 4 KiB page, and print the faults from getrusage.
Done when: about 16,387 faults and 35 faults on a machine with THP set to madvise or always, with a note if your numbers differ.
Hint
Align the start with mmap of an extra 2 MiB and a round-up.
Medium 2: Bloat (45 minutes)
Map a 2 MiB aligned region, write one byte, and print the resident size before and after, with and without MADV_HUGEPAGE.
Done when: growth of 4 kB without and 2,048 kB with huge pages, and a sentence on what this means for a process with 1,000 sparse regions.
Hint
Read the second number of /proc/self/statm, in pages.
Medium 3: Check the counters (45 minutes)
Read thp_fault_alloc and thp_fault_fallback from /proc/vmstat before and after Medium 1. Report the difference.
Done when: thp_fault_alloc rose by about 32 for the huge run, and thp_fault_fallback by 0 (or a number with an explanation).
Hint
Take the numbers with grep thp_fault /proc/vmstat.
Hard
Hard 1: The pointer-chase test (90 minutes)
Build the pointer-chasing test with regions of 2, 16, 128 and 256 MiB, aligned to 2 MiB, once with and once with no huge pages. Report the time per access.
Done when: a table like section 5, with a clear gain at 2 MiB, and a short explanation of what happens in your results at the larger sizes.
Hint
Compile with -O2, and time 20 million pointer reads.
Hard 2: A sparse structure (90 minutes)
Write a program that allocates 1,000 separate 2 MiB aligned regions and writes one byte in each. Report the resident size with and without MADV_HUGEPAGE.
Done when: about 4 MB against about 2,000 MB (2 GB) of resident memory, and a note on what THP always would do to a program like this.
Hint
Use a loop with mmap for each region, and madvise on each.
Hard 3: A real workload (90 minutes)
Take a program of your own that uses a large array or hash table (at least 256 MiB). Run it with THP set to never and to always (as the administrator) or with madvise. Report run time, resident size and AnonHugePages.
Done when: a table of the three measurements for both settings and a recommendation that the data supports.
Hint
Write to /sys/kernel/mm/transparent_hugepage/enabled as the administrator.