CSE Roadmap

Core tier · about 3 hours · 7 parts

Part 5 of 7 · 3 min read

pids and cpu limits

Think of a theme park ride that takes at most five riders at once, and a turnstile that lets the group in for half of each hour. The first limit is a count, and the second is a share of time. The analogy stops in one place: a ride waits politely, and a process over its CPU quota is stopped for the rest of the period, which the program feels as slow progress.

In plain words: The pids controller caps the number of processes and threads in the group, which stops a fork bomb (a program that forks without end). A fork that would pass the limit fails with EAGAIN, "Resource temporarily unavailable". The cpu controller can cap the group's time. With a period of 100,000 microseconds (100 ms) and a quota of 50,000, the group may run for 50 ms of CPU time in each 100 ms, which is half a core. When the quota is used up, every process of the group is throttled (stopped) until the next period begins.

A worked example. Two more programs.

#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <sys/wait.h>
#include <unistd.h>

int main(void) {
    int ok = 0;
    for (int i = 0; i < 50; i++) {
        pid_t p = fork();
        if (p == 0) { alarm(2); pause(); _exit(0); }
        if (p < 0) { printf("fork %d failed: %s\n", i + 1, strerror(errno)); break; }
        ok++;
    }
    printf("children started: %d\n", ok);
    fflush(stdout);
    while (wait(NULL) > 0) {}
    return 0;
}
#include <stdio.h>
#include <time.h>

int main(void) {
    struct timespec a, b;
    clock_gettime(CLOCK_MONOTONIC, &a);
    while (1) {
        clock_gettime(CLOCK_MONOTONIC, &b);
        if (b.tv_sec - a.tv_sec >= 3) break;     /* spin for 3 s of wall time */
    }
    printf("cpu time used: %.2f s of 3.00 s wall\n", (double)clock() / CLOCKS_PER_SEC);
    return 0;
}

With pids.max set to 5, and with a quota of 50,000 out of 100,000:

fork 5 failed: Resource temporarily unavailable
children started: 4

cpu time used: 1.40 s of 3.00 s wall         (quota 50,000 of 100,000)
nr_periods 28
nr_throttled 28
throttled_time 1376285723
cpu time used: 2.96 s of 3.00 s wall         (no group)

The group limit of 5 counts the program itself, so 4 children fit, and the fifth fork failed. The CPU program spun for 3 seconds of wall time and used only 1.40 s of CPU time inside the group, about 47 per cent (1.40 / 3.00), close to the 50 per cent quota, and 2.96 s (99 per cent) outside it. The file cpu.stat confirms: of 28 periods, 28 were throttled, and the total time spent waiting was 1,376,285,723 ns, or 1.38 s. The throttled time and the used time together are close to the 3 s of wall time (1.40 + 1.38 = 2.78 s, and the rest is the start and the end of the run).

The key lines. A CPU quota is a limit, not a share: it holds even when the machine is idle. A share (cpu.shares in version 1, cpu.weight in version 2) only matters when processes compete.

CPU quota and period

A CPU quota is the amount of processor time a group may use in each period. A quota of 50,000 microseconds in a period of 100,000 gives half of one core. When the group uses up its quota, its processes are throttled until the next period starts.

Example: cpu.cfs_quota_us = 50000 with cpu.cfs_period_us = 100000 gave 1.40 s of CPU in 3 s of wall time.

Check yourself: 1. Why did only 4 children start with pids.max 5? 2. What fraction of a core is a quota of 25,000 in a period of 100,000? 3. What does nr_throttled count?