CSE Roadmap

Core tier · about 3 hours · 6 parts

Part 4 of 6 · 3 min read

Measurement

In plain words: A switch can be measured by forcing two processes to take turns. Process P writes one byte into a pipe and then waits to read an answer. Process Q waits to read, then writes the byte back. Each process must wait for the other, so every step is a switch. Both are tied to the same processor with sched_setaffinity, so that they cannot run in parallel on two processors.

#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>
#include <sched.h>
#include <time.h>
#include <sys/wait.h>

int main(void) {
    enum { N = 200000 };
    int p2c[2], c2p[2];
    if (pipe(p2c) || pipe(c2p)) { perror("pipe"); return 1; }
    cpu_set_t set; CPU_ZERO(&set); CPU_SET(0, &set);
    sched_setaffinity(0, sizeof set, &set);       /* both on CPU 0 */
    pid_t pid = fork();
    char b = 'x';
    if (pid == 0) {
        for (int i = 0; i < N; i++)
            if (read(p2c[0], &b, 1) != 1 || write(c2p[1], &b, 1) != 1) _exit(1);
        _exit(0);
    }
    struct timespec t0, t1;
    clock_gettime(CLOCK_MONOTONIC, &t0);
    for (int i = 0; i < N; i++)
        if (write(p2c[1], &b, 1) != 1 || read(c2p[0], &b, 1) != 1) return 1;
    clock_gettime(CLOCK_MONOTONIC, &t1);
    wait(NULL);
    double ns = (t1.tv_sec - t0.tv_sec) * 1e9 + (t1.tv_nsec - t0.tv_nsec);
    printf("%d round trips, %.0f ns per round trip, about %.0f ns per switch\n",
           N, ns / N, ns / N / 2);
    return 0;
}

Three runs on the test machine (two processors, a virtual machine):

200000 round trips, 1875 ns per round trip, about 937 ns per switch
200000 round trips, 1832 ns per round trip, about 916 ns per switch
200000 round trips, 1930 ns per round trip, about 965 ns per switch

One round trip has two switches, one to Q and one back to P, so each switch is about half of 1,875 ns, which is 937 ns. The figure is an upper bound, because it also contains four system calls (two reads and two writes), each of about 87 ns, and 4 × 87 = 348 ns. Subtracting that from 1,875 leaves 1,527 ns for two switches, or about 760 ns each. So a context switch costs a little under a microsecond on this machine. Other machines give other numbers, often from 1 to 5 microseconds on real hardware, and the result varies by a few per cent between runs.

A second program confirms that the switches really happened. It runs the same ping-pong for 1,000 round trips and prints its own voluntary-switch counter from /proc/self/status (page 2). The parent showed 572. The count is below 1,000 because sometimes the answer was already in the pipe when the parent looked, so the parent did not have to wait.

Read it as

One microsecond is 1,000 ns. At a rate of one switch per millisecond, a switch costs 0.1 per cent of the time. At one switch per 10 microseconds it costs 10 per cent.

Check yourself: 1. Why must both processes run on the same processor in this test? 2. Why is the figure per round trip divided by two? 3. Why is 937 ns an upper bound?