CSE Roadmap

Core tier · about 3 hours · 6 parts

Part 5 of 6 · 3 min read

Pool settings

Pool size, queue size and rejection.

Sizing and overload

JCIP describes the settings of ThreadPoolExecutor. The core size is the number of threads kept ready. The maximum size is the most that can run at once, and extra threads are created only when the work queue is full. The work queue holds tasks waiting for a thread. Idle extra threads are removed after a keep-alive time.

The standard factories choose these for you. newFixedThreadPool has the same core and maximum size, so it limits the number of tasks running at once. JCIP recommends it when you must limit concurrency to protect a resource, such as a server that would otherwise be overwhelmed. newCachedThreadPool starts a thread whenever none is free and has no upper limit. EJ says that this is fine for a light load, and a poor choice for a heavily loaded server, where a fixed pool or a configured ThreadPoolExecutor is better.

JCIP also warns that both limits matter for stability: bound the queue, or overload will use up memory, and bound the threads, or it will exhaust the machine.

When a bounded queue fills, the saturation policy decides what to do. JCIP lists four. Abort throws an exception to the caller. Discard drops the new task quietly. Discard-oldest drops the task at the head of the queue. Caller-runs makes the submitting thread run the task, which slows the producer naturally.

A test shows abort in action. A pool of one thread has a queue of one place. Four slow tasks are submitted.

ThreadPoolExecutor tp = new ThreadPoolExecutor(1, 1, 0, TimeUnit.SECONDS,
        new ArrayBlockingQueue<>(1), new ThreadPoolExecutor.AbortPolicy());

The first runs on the thread, the second waits in the queue, and the third and fourth are rejected. The output reads "accepted 2 rejected 2". The caller can retry, tell the user the system is busy, or switch to caller-runs. Which choice is right depends on whether dropping work is acceptable, which is a design decision to state in a round.

There is no single right pool size. For tasks that mostly compute, a size near the number of CPU cores is a starting point. For tasks that mostly wait on a network, more threads can be useful. Measure, then adjust.

Common mistake

Using an unbounded queue and an unlimited pool and assuming it will cope. Under overload, memory or threads run out. Choose limits and a saturation policy on purpose.

Interview angle

Answer in your head first, then open it.

What is the producer-consumer pattern?

Producers put work on a shared queue and consumers take it off, so each side runs at its own pace and neither knows about the other.

Why use a bounded queue?

It limits memory and slows the producers when consumers fall behind.

Why a thread pool and not a thread per task?

Threads are costly to create and unlimited ones can overload the machine. A pool reuses a fixed number.

What happens when the queue is full?

The saturation policy decides: reject, drop, or run the task in the caller's thread.

Check yourself: 1. What is the difference between core size and maximum size? 2. Which saturation policy slows the producer without losing work? 3. In the test, why are two of the four tasks rejected?

Words you will meet

Word Meaning
Blocking queue A queue whose put and take wait for room or items
Back pressure Slowing producers when consumers fall behind
Poison pill A marker item that tells a consumer to stop
Future A handle for a result that may not be ready yet
Saturation policy The rule for a task that arrives when the queue is full