Part 3 of 7 · 4 min read
Thread pool
In plain words: A thread pool creates a fixed number of worker threads once, at the start. Requests do not create threads. They are put into a work queue, and each idle worker takes the next one, runs it, and goes back for another. A bank with four tellers and one line of customers works this way. The analogy stops in one place: a teller who is busy cannot be asked to speak to two customers at once, and a worker is the same.
The parts:
- A queue of tasks, with a fixed size (a ring buffer here, in an array).
- A mutex to protect the queue, since several threads use it.
- Condition variables so a worker can sleep when the queue is empty, and a producer can wait when it is full (next section).
- A set of worker threads in a loop: take a task, run it, repeat.
- A shutdown flag, so the workers can end when no more work will come.
Here is a complete pool in two pieces. The first piece is the queue and the function that adds a task:
#define QSIZE 16
#define NWORKERS 4
#define NTASKS 20
struct task { int id; };
static struct task queue[QSIZE];
static int head, tail, count, done;
static pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t not_empty = PTHREAD_COND_INITIALIZER;
static pthread_cond_t not_full = PTHREAD_COND_INITIALIZER;
static long results[NTASKS];
static void submit(int id) {
pthread_mutex_lock(&lock);
while (count == QSIZE) pthread_cond_wait(¬_full, &lock);
queue[tail].id = id; tail = (tail + 1) % QSIZE; count++;
pthread_cond_signal(¬_empty);
pthread_mutex_unlock(&lock);
}
The second piece is the worker and main. Each task adds the numbers 1 to id:
static void *worker(void *arg) {
(void)arg;
for (;;) {
pthread_mutex_lock(&lock);
while (count == 0 && !done) pthread_cond_wait(¬_empty, &lock);
if (count == 0 && done) { pthread_mutex_unlock(&lock); return NULL; }
struct task t = queue[head]; head = (head + 1) % QSIZE; count--;
pthread_cond_signal(¬_full);
pthread_mutex_unlock(&lock);
long s = 0;
for (int i = 1; i <= t.id; i++) s += i;
results[t.id - 1] = s; /* own slot: no lock needed */
}
}
int main(void) {
pthread_t w[NWORKERS];
for (int i = 0; i < NWORKERS; i++) pthread_create(&w[i], NULL, worker, NULL);
for (int id = 1; id <= NTASKS; id++) submit(id);
pthread_mutex_lock(&lock); done = 1;
pthread_cond_broadcast(¬_empty); pthread_mutex_unlock(&lock);
for (int i = 0; i < NWORKERS; i++) pthread_join(w[i], NULL);
long total = 0;
for (int i = 0; i < NTASKS; i++) total += results[i];
printf("%d tasks done by %d workers, sum of results = %ld\n", NTASKS, NWORKERS, total);
return 0;
}
Three runs printed 20 tasks done by 4 workers, sum of results = 1540 each time. Check by hand: task n produces n(n+1)/2, and the sum for n from 1 to 20 is 1,540 (a separate calculation agrees). Which worker took which task changes from run to run, but the total never does, because each task writes only its own slot of results and the main thread reads them after all workers are joined.
Notice what the worker does with the lock: it holds it only while taking a task from the queue, never while running the task. If it held the lock during the work, only one worker could work at a time.
Thread pool
A thread pool is a fixed set of worker threads, created once, that repeatedly take tasks from a shared work queue and run them, so that tasks do not each need a new thread.
Example: a web server with 16 workers and a queue of waiting requests handles any number of requests with 16 threads.
Check yourself: 1. What does a worker do when the queue is empty? 2. Why does the worker release the lock before running the task? 3. Why is each result written to its own array slot?