Part 6 of 7 · 3 min read
Tools on a real system
Watching memory pressure on Linux
In plain words: A real machine shows its memory state in a few places. The tool vmstat 1 prints one line each second. The columns si and so give the amount of memory read from swap space and written to swap space in each second. A machine that is not short of memory shows 0 and 0 there. A machine that thrashes shows large numbers in both columns at the same time, together with a high wa (the share of time the processors wait for input and output).
The file /proc/vmstat holds counters since boot. The ones to watch are pgfault (all faults), pgmajfault (faults that needed storage), pswpin and pswpout (pages read from and written to swap space). Two readings taken a few seconds apart give a rate. On the test machine, pgmajfault was 1,759 against 7,014,886 for pgfault: most faults are minor, as page 7 explained.
A newer view is the pressure stall information. On Linux kernels from version 4.20, a file such as /proc/pressure/memory has lines like these (all values here are zero on an idle machine):
some avg10=0.00 avg60=0.00 avg300=0.00 total=0
full avg10=0.00 avg60=0.00 avg300=0.00 total=0
A "some" line reports the share of time in which at least one task was stalled waiting for memory (for example for a major fault), and a "full" line the share of time in which all tasks that could run were stalled. The averages cover the last 10, 60 and 300 seconds, and total counts the stall time in microseconds. A machine that thrashes shows a "full" value that climbs, and this gives an early warning before the out-of-memory killer acts.
Pressure stall information
Pressure stall information (PSI) is a Linux facility that reports, in files such as /proc/pressure/memory, how much of the time tasks were stalled while waiting for memory, so that memory shortage can be seen as a number.
Example: a "some" line with avg10=35.00 says that for 35% of the last ten seconds at least one task was waiting for memory.
Check yourself:
1. What do large si and so values at the same time suggest?
2. Which counter in /proc/vmstat counts faults that needed storage?
3. What does a rising "full" value in the pressure file mean?
Words you will meet
| Word | Meaning |
|---|---|
| Equal allocation | Every process gets the same number of frames |
| Proportional allocation | Frames are shared in proportion to process size |
| Local replacement | The victim comes from the faulting process |
| Global replacement | The victim may come from any process |
| Thrashing | Most time goes to page faults |
| Locality | The set of pages a program uses heavily for a period |
| Working set | The distinct pages used in the last Δ references |
| WSS | The size of the working set |
| Suspend | Take all frames from a process and stop running it |
| PFF | Control of frames by the page-fault rate |
| Out-of-memory killer | The Linux part that ends a process when memory is exhausted |
| PSI | Linux numbers for the time spent stalled on memory |
Quick notes
- Global replacement is a page replacement policy in which the victim is chosen from all the frames in memory, so a process can take a frame from another process, while.
- Thrashing is the state in which processes spend more time handling page faults than doing useful work, because the frames they have are too few to hold the pages they.
- The working set of a process at a time t, for a window of Δ references, is the set of distinct pages that the process touched in its last Δ.
- Page-fault frequency (PFF) control is a method of managing frames in which the kernel measures the fault rate of each process and gives frames to a process whose rate is.
- Pressure stall information (PSI) is a Linux facility that reports, in files such as
/proc/pressure/memory, how much of the time tasks were stalled while waiting for memory, so that memory shortage can be seen as a number.