BrewlearnSizing thread pools in Java
CPU-bound vs I/O-bound work, rejection policies, bounded queues.
CPU-bound work
For pure computation, use about one thread per core (Runtime.getRuntime().availableProcessors()). Only one thread per core can run at a time, so 10× more threads doesn't make it 10× faster. They just compete and add context-switch overhead.
I/O-bound work
Threads that mostly wait (on databases, networks, disks) leave cores idle, so you need more of them: roughly cores × (1 + wait time / compute time). Or skip the math for blocking I/O and use virtual threads.
int cores = Runtime.getRuntime()
.availableProcessors();
// 90 ms waiting per 10 ms of CPU:
int threads = cores * (1 + 90 / 10);Your turn
A 4-core server runs tasks that wait 50 ms on a remote API for every 10 ms of CPU work. About how many threads keep the CPUs busy?
About 4About 24About 400
Show the answer
4 × (1 + 50/10) = 24. Each thread uses the CPU only one-sixth of the time, so you need about six threads per core to keep every core busy.
Unbounded queues hide overload
newFixedThreadPool(10) uses an unbounded LinkedBlockingQueue. Bursts pile up in memory until OutOfMemoryError. A cached pool just creates unbounded threads instead. Fix: a ThreadPoolExecutor with a bounded queue and a rejection policy.
new ThreadPoolExecutor(
10, 10, 0, TimeUnit.SECONDS,
new ArrayBlockingQueue<>(1_000),
new ThreadPoolExecutor.CallerRunsPolicy());Rejection policies
When the pool and its queue are both full: AbortPolicy (default) throws RejectedExecutionException. CallerRunsPolicy makes the submitting thread run the task itself, slowing producers down: back-pressure. DiscardPolicy silently drops the new task. DiscardOldestPolicy drops the oldest queued task, then retries.
The pool that deadlocks itself
A fixed pool of 4 threads runs tasks that each submit a subtask to the same pool, then block on its future.get(). Under load everything hangs at 0% CPU. Why?
Think about it, then reveal the answer
All 4 threads are blocked waiting for subtasks that sit in the queue behind them, and those subtasks need a free thread to run. That's thread-starvation deadlock. Use separate pools for parent and child tasks, or don't block inside pooled tasks.
Bulkheads
Production systems often give each downstream dependency its own bounded pool, like watertight compartments in a ship. If the payments API hangs, only its pool fills up; search and checkout keep their threads. Pair that with timeouts and CallerRuns or rejection, and overload becomes visible instead of fatal.
Key takeaways
- CPU-bound: about the number of cores
- I/O-bound: cores × (1 + wait/compute) — or virtual threads
- Unbounded queues hide overload until OutOfMemoryError
- Rejection policies: Abort (default), CallerRuns, Discard, DiscardOldest
HikariCP's pool-sizing guide suggests starting near (cores × 2) + effective spindle count database connections, often far fewer than people expect.
Practice questions
An 8-core server runs tasks that wait 90 ms on a database for every 10 ms of CPU work. Roughly how many threads keep the CPUs busy?
- About 80
- About 8
- About 9
- About 800
Check your answer
About 80. 8 × (1 + 90/10) = 80. Each thread uses the CPU only 10% of the time, so you need about ten times as many threads as cores.
A fixed pool of 4 threads runs tasks that each submit a subtask to the SAME pool and then block on its future.get(). Under load, everything hangs with 0% CPU. Why?
- All 4 threads wait for subtasks stuck in the queue behind them — a pool-induced deadlock
- Futures can't be used from inside pool threads
- The full queue silently drops the subtasks
- get() busy-spins and starves the subtasks
Check your answer
All 4 threads wait for subtasks stuck in the queue behind them — a pool-induced deadlock. This is thread-starvation deadlock: the subtasks need a free thread, but every thread is blocked waiting for them. Use separate pools, or don't block inside pooled tasks.