CSE Roadmap

Core tier · about 4 hours · 7 parts

Part 6 of 7 · 4 min read

Event loops

Event loops

In plain words: For tasks that mostly wait, there is a second approach that needs no pool. A single thread asks the kernel: "tell me when any of these descriptors is ready." While nothing is ready, the thread sleeps. When something is ready, it handles that event for a short time and asks again. A single waiter at a restaurant who checks which tables have raised a hand, and serves only those, is the picture. The analogy stops where the waiter must also do the cooking: an event loop must never do long work inside a handler, or all other clients wait.

The call is poll (or epoll on Linux for very many descriptors). This program creates two pipes fed by a child process, one writing after 0.1 s and one after 0.3 s, and one thread serves both:

struct pollfd fds[2] = { { a[0], POLLIN, 0 }, { b[0], POLLIN, 0 } };
int open_fds = 2;
while (open_fds > 0) {
    poll(fds, 2, -1);                    /* sleep until some descriptor is ready */
    for (int i = 0; i < 2; i++) {
        if (fds[i].revents & POLLIN) {
            char buf[32]; ssize_t n = read(fds[i].fd, buf, sizeof buf - 1);
            if (n > 0) { buf[n] = 0; printf("one thread got: %s", buf); }
        }
        if (fds[i].revents & POLLHUP) { close(fds[i].fd); fds[i].fd = -1; open_fds--; }
    }
}
one thread got: from A
one thread got: from B

One thread served two sources in the order they became ready. It never blocked on the second source while waiting for the first, and it used no lock, because there is only one thread and no data is shared.

Thread pool Event loop
Parallel on several processors Yes No, one thread (run one loop per processor for more)
Code style Each task is ordinary code that may block Handlers must not block, so code is split into pieces
Locks needed Yes, for shared data Few or none
Good for Computing, blocking calls Very many connections that mostly wait

Real servers often combine them: one event loop per processor, plus a pool for slow or blocking work. Nginx and Node.js are built around event loops, while many Java servers use pools.

Event loop

An event loop is a loop in a single thread that waits, with a call such as poll or epoll, until one of many watched descriptors is ready, handles that event with a short function, and then waits again.

Example: a chat server that reads from thousands of connections in one thread, sleeping in epoll_wait between messages, uses an event loop.

Interview angle

Answer in your head first, then open it.

Why use a thread pool instead of a thread per request?

It avoids the creation cost, limits the number of threads and so the memory and switching cost, and gives steady performance under load.

How do workers wait for work?

On a condition variable, inside a while loop that rechecks the queue.

How big should a pool be?

About the number of processors for CPU-bound work; more for I/O-bound work, by the ratio of wait time to compute time.

What is an event loop?

One thread waits for readiness events from many sources and handles each briefly.

When is an event loop better than threads?

When there are very many connections that mostly wait.

Check yourself: 1. Why must a handler in an event loop not block? 2. Can one event loop use several processors? 3. Name one server built on an event loop.

Words you will meet

Word Meaning
Thread-per-request A new thread for every request
Thread pool A fixed group of workers taking tasks from a queue
Work queue The list of tasks waiting for a worker
Condition variable A way to sleep until a condition may be true
Busy waiting Looping on a check and wasting processor time
Event loop One thread waiting on many sources of events

Quick notes

  • The thread-per-request model is a server design in which a new thread is created for each incoming request and ends when the request is finished.
  • A thread pool is a fixed set of worker threads, created once, that repeatedly take tasks from a shared work queue and run them, so that tasks do not each need a new thread.
  • A condition variable is a synchronisation object that lets a thread, while holding a mutex, wait until another thread signals that some condition may have become true.
  • A CPU-bound task is one whose run time is set by the processor's speed, because it spends almost all its time computing.
  • An event loop is a loop in a single thread that waits, with a call such as poll or epoll, until one of many watched descriptors is ready, handles that event with a short function, and then waits again.