CSE Roadmap

Core tier · about 5 hours · 7 parts

Part 6 of 7 · 3 min read

Memory limits

Living with limits

Programs see memory failure in two ways, and they need different care.

A failed request. When a limit applies to address space, malloc returns NULL. The shell command ulimit -v 524288 limits the address space of its programs to 512 MiB (the value is in KiB). The test program that allocates 10 MiB at a time printed touched 500 MiB and then malloc failed at 500 MiB. The failure came at the request, 12 MiB below the limit, because the C library and the program image use the rest. A program that checks the result can free a cache, report an error or stop in an orderly way. A program that does not check will crash on the null pointer.

A kill. When the shortage comes at a page touch, as in section 3, nothing can be caught. The only protection is to keep the program's memory below the limit that applies to it, and to find out before the machine does. The limits that matter:

Limit Set by What happens at the limit
RLIMIT_AS (ulimit -v) the shell or the program malloc and mmap fail with ENOMEM
cgroup memory limit the container runtime, the administrator pages are reclaimed, then the OOM killer ends a process in the group
machine memory hardware the global OOM killer

In plain words: Good practice follows. Check every malloc. Put bounds on caches and queues, so that growth has a cap. Monitor the resident size and the pressure counters (page 9 of Level 6 and page 8 of this level). Size memory limits with a margin, and test what happens at the limit before production does.

Control group memory limit

A control group (cgroup) memory limit is a cap on the memory used by all the processes in a group, set by the administrator or a container runtime, beyond which the kernel reclaims pages from the group and finally kills a process inside it.

Example: a container started with a 200 MiB limit is killed when it touches more, and the rest of the machine is not affected.

Check yourself: 1. What does malloc return when ulimit -v is exceeded? 2. Why can a program not catch an OOM kill? 3. Name two ways to keep the effect of a memory shortage inside one service.

Words you will meet

Word Meaning
Swap space Disk space for anonymous pages
Swapping out and in Writing a page to swap and reading it back
vm.swappiness How willingly the kernel swaps anonymous pages
Overcommit Accepting requests that add up to more than the memory
CommitLimit The total allowed in strict mode
Committed_AS The total promised now
OOM killer Ends a process when memory is exhausted
oom_score_adj A per-process shift of the OOM score
SIGKILL The signal that cannot be caught
RLIMIT_AS The limit on address space (ulimit -v)
Control group A set of processes with shared limits

Quick notes

  • Swap space is an area of disk, a partition or a file, to which the kernel writes the contents of anonymous pages (those with no file behind them) so that.
  • Overcommit is the kernel's practice of accepting requests for more memory, in total, than the machine can supply, on the expectation that programs will touch only part of it, and.
  • The OOM killer is the part of the Linux kernel that, when memory is exhausted and no frame can be freed, chooses a process and ends it with SIGKILL so that its frames can be reused.
  • oom_score_adj is a number from −1000 to +1000, set per process in /proc/PID/oom_score_adj, that is added to the process's OOM score, so that a high value makes the OOM killer choose it sooner and −1000 makes it exempt.
  • A control group (cgroup) memory limit is a cap on the memory used by all the processes in a group, set by the administrator or a container runtime, beyond which.