Part 2 of 7 · 2 min read
Looking
In plain words: A doctor reads a pulse, a temperature and a blood pressure before saying what is wrong. A scheduling problem needs the same few readings first. The analogy stops in one place: a patient's readings are steady, while a machine's change many times in a second.
The first reading is the class and priority of each task. ps shows them:
$ ps -eo pid,ni,pri,cls,psr,stat,comm
PID NI PRI CLS PSR STAT COMMAND
1 0 19 TS 0 SLl process_api
4 -20 39 TS 0 I< kworker/R-rcu_gp
NI is the nice value, CLS the scheduling class (TS is the normal time-sharing class, FF is real-time FIFO, RR is real-time round robin), and PSR is the processor the task last ran on.
The second reading is the switch counts (page 1): voluntary_ctxt_switches and nonvoluntary_ctxt_switches in /proc/<pid>/status. Many forced switches mean a task that wants more processor time than it gets. Many voluntary switches mean a task that waits often.
The third is the kernel's own bookkeeping in /proc/<pid>/sched, which shows se.vruntime, se.sum_exec_runtime, se.load.weight, nr_switches and se.nr_migrations (page 5 and page 6 used all of them).
The fourth is the load of the whole machine. vmstat 1 prints a line a second. Its first column, r, is the number of tasks that are runnable (running or waiting for a processor). Four spinning loops on the 2-processor machine gave:
procs ---------cpu--------
r b ... us sy id wa
4 0 ... 100 0 0 0
4 0 ... 98 0 0 0
Four runnable tasks on two processors: r is twice the number of processors, and id (idle) is 0. Two tasks wait for a processor all the time.
uptime and /proc/loadavg give the load average, three numbers for the last 1, 5 and 15 minutes. They change slowly. In the test, the 1-minute figure was only 1.16 after the four loops had run for a few seconds, and it takes minutes to approach 4. A load average above the number of processors suggests a backlog, and a rising 1-minute figure with a lower 15-minute figure suggests that the load has just started.
Load average
The load average is an exponentially damped average, over 1, 5 and 15 minutes, of the number of tasks that are runnable or are waiting in uninterruptible sleep (usually for disk I/O), reported by uptime and in /proc/loadavg.
Example: a 1-minute load average of 4.0 on a 2-processor machine means that, on average, about two tasks more than the machine can run were waiting.
Check yourself:
1. What does the r column of vmstat count?
2. Why was the 1-minute load average only 1.16 after a few seconds of four busy loops?
3. Which file shows a task's vruntime?