CSE Roadmap

Core tier · about 5 hours · 7 parts

Part 4 of 7 · 2 min read

Big requests

In plain words: A big request is different. A block of 200 KB taken from the heap would leave a large hole when freed, and the heap could shrink only from the top. glibc therefore serves large requests with their own mmap region. When the block is freed, munmap returns it to the kernel at once. The size limit is called the mmap threshold, and its default is 128 KiB (131,072 bytes).

The measured results of requests in a fresh process:

Request Where the block went
1,000 bytes the heap
100,000 bytes the heap
132,000 bytes the heap
200,000 bytes its own mmap region

The 132,000-byte request is above the 128 KiB threshold, and still came from the heap. The reason is that glibc first tries the free space it already has. The first malloc had enlarged the heap by 135,168 bytes, and that space was big enough for 132,000 bytes, so no new mapping was needed. The threshold matters when the heap cannot serve the request. The 200,000-byte request did not fit, and strace shows the call mmap(NULL, 200704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0). The 200,704 is the request plus the header, rounded up to whole pages.

A second effect is the dynamic threshold. strace on a loop of five calls malloc(1 << 20) followed by free shows mmap and munmap for the first round only. After the first free of a mapped block, glibc raises the threshold to that block's size, on the guess that the program will keep asking for blocks of that size. The later rounds are served from the heap. The cost is visible in a timing: 1 MiB malloc and free took 0.0 µs per pair with the dynamic threshold, and 6.4 µs per pair when a fixed threshold of 131,072 forced a mapping each time (the loop touched one byte of each block).

mmap threshold

The mmap threshold is the request size above which the allocator, when the heap cannot serve a request, gets a separate region from the kernel with mmap and later returns it with munmap.

Example: with the default of 128 KiB, a 200,000-byte request that finds no room in the heap gets its own mapping.

Common mistake

"Every malloc above 128 KiB calls mmap" is only roughly true. The allocator uses free heap space first, and glibc raises the threshold as a program runs. For a fixed behaviour, set mallopt or the environment variable MALLOC_MMAP_THRESHOLD_.

Check yourself: 1. Why does a 200,000-byte block get its own region? 2. Why did the 132,000-byte request stay in the heap? 3. What does the dynamic threshold do after the first free of a mapped block?