Part 2 of 7 · 2 min read
Thread per task
Think of it this way. A restaurant that hires a new waiter for every customer and sends the waiter home after one meal wastes the time spent hiring. The analogy stops in one place: hiring a thread is far cheaper than hiring a person, but the idea of the waste is the same.
In plain words: The simplest server design starts a new thread for each request and ends it when the request is done. It is easy to write, since each thread handles one request from start to end with ordinary code. It has three costs:
- Creation time. Each
pthread_createandpthread_jointakes tens of microseconds (page 1 measured about 33 µs on the test machine). For a task of a few microseconds, the start-up is the biggest cost. - Memory. Each thread needs a stack. The default on Linux is 8 MB of address space (
ulimit -s). Pages are only given real memory when touched, but thousands of threads still use large amounts of address space and kernel memory. - No upper limit. If a million requests arrive, the program tries to create a million threads. The system has limits (the test machine showed 32,018 for
ulimit -u), and long before that, the machine spends its time switching between threads (Level 1, page 5).
Thread-per-request model
The thread-per-request model is a server design in which a new thread is created for each incoming request and ends when the request is finished.
Example: a server that calls pthread_create for each client connection and lets that thread read the request and send the reply uses this model.
Check yourself: 1. Name the three costs of starting a thread for every task. 2. What happens to a server that gets a million requests at once under this model? 3. Why is the start-up cost most painful for very short tasks?