How do you know your Node.js endpoint is CPU-bound?
If your service runs hashing, image resizing, large JSON or XML parsing, compression, or server-side rendering inside request handlers, check whether that work is blocking the event loop before you add threads. Threads fix one specific problem, and they add overhead if that is not your problem.
Measure event loop delay
perf_hooks.monitorEventLoopDelay() returns a histogram that samples how late the event loop runs its timers. Enable it at startup and export percentile(99) and max to your metrics system. The values are in nanoseconds, so convert them before you graph them. On a healthy service, delay stays near the sampling resolution. When a synchronous function holds the main thread, every queued callback waits behind it, and p99 delay climbs to roughly the duration of your longest synchronous task.
Tell CPU-bound from I/O-bound
Read delay together with process CPU:
- CPU-bound: one core is pinned near 100%, p99 event loop delay is high, and unrelated routes slow down too. A health check that does no work still takes hundreds of milliseconds. One expensive request is hurting everyone else.
- Slow I/O: CPU is low, event loop delay stays flat, and only the routes that call the slow database or upstream service are slow. Threads will not help here.
Find the hot function
Run the process under node --cpu-prof while it handles representative load. On exit, Node writes a .cpuprofile file that you can open in Chrome DevTools. Alternatively, attach the inspector with --inspect and record a profile live. Look for wide, flat frames under your handler, such as a hash, a codec, or a parser. Those frames are the code to move off the main thread.
// Run under load with: node --cpu-prof server.js
// Ctrl+C triggers a clean exit so Node writes the .cpuprofile file.
const http = require('node:http');
const crypto = require('node:crypto');
const { monitorEventLoopDelay } = require('node:perf_hooks');
// Sample event loop lateness every 20 ms.
const histogram = monitorEventLoopDelay({ resolution: 20 });
histogram.enable();
// Histogram values are nanoseconds; convert before reporting.
const toMs = (ns) => (ns / 1e6).toFixed(1);
setInterval(() => {
// In production, export these to your metrics system instead of logging.
console.log(`loop delay p99=${toMs(histogram.percentile(99))}ms max=${toMs(histogram.max)}ms`);
histogram.reset(); // report per-interval values, not lifetime values
}, 5000).unref();
http.createServer((req, res) => {
if (req.url === '/hash') {
// Synchronous key derivation holds the main thread for its full duration.
const key = crypto.pbkdf2Sync('example-password', 'example-salt', 500000, 64, 'sha512');
res.end(key.toString('hex'));
return;
}
// /health does no work, yet it slows down whenever /hash is running.
res.end('ok');
}).listen(3000);
process.on('SIGINT', () => process.exit(0));
A minimal server that reports p99 and max event loop delay with perf_hooks.monitorEventLoopDelay while a synchronous pbkdf2Sync route blocks the main thread; run it under node --cpu-prof to capture the hot function.
Who should use worker threads with Piscina?
Piscina fits teams that run a Node.js HTTP or RPC service where the measurements above point at a small number of routes doing synchronous CPU work, and where that work has to finish before the response is sent. Typical cases:
- API services with one or two heavy routes. Thumbnail generation, password hashing with a synchronous library, PDF or report rendering, schema validation of large documents, or server-side rendering, sitting next to many cheap routes that suffer when the heavy ones run.
- Services with spare CPU in their allocation. The container or VM has more than one CPU's worth of quota, but the process only ever keeps one core busy because all JavaScript runs on the main thread.
- Teams that want to stay in one codebase. The CPU-heavy function is already written in JavaScript or TypeScript, and the team would rather move it to another thread than rewrite it as a native addon or split it into a separate service.
It is the wrong tool for:
- I/O-bound services. If event loop delay is flat and CPU is low, workers add memory and complexity without making anything faster.
- Sub-millisecond tasks. Dispatch and cloning overhead can exceed the work itself. The estimate later in this article shows why.
- Single-CPU allocations. With a quota of one CPU, a worker still keeps the event loop responsive by taking the heavy task off the main thread, but it competes with the main thread for the same CPU time, so total throughput does not rise.
- Work that outlives the request. Jobs that run for minutes, or that must survive a process restart, belong in a job queue.
How do worker threads and Piscina work, and why doesn't async I/O help?
Node's worker_threads module lets one process run JavaScript on several OS threads. Each Worker gets its own V8 isolate, its own heap, and its own event loop. Workers share no JavaScript objects with the main thread. They communicate by message passing, and arguments and results are copied with the structured clone algorithm. The exceptions are ArrayBuffers you explicitly transfer and SharedArrayBuffers. Starting a worker means booting a new isolate, which costs startup time and memory. That cost is why you pool workers instead of spawning one per request.
Piscina is that pool. It starts a set of workers that all load the same task file and keeps them alive. It queues incoming tasks and hands each one to an idle worker. Each call returns a promise that settles when the worker returns.
This is different from libuv's internal thread pool, which is small and fixed-size by default and configured with UV_THREADPOOL_SIZE. That pool runs native C++ work for fs, dns.lookup, zlib, and the async crypto functions such as pbkdf2. It never runs your JavaScript.
That limitation is also why async/await doesn't help with CPU-bound code. Marking a function async doesn't move it to another thread. Every line between two awaits still runs synchronously on the main thread. A tight hashing loop inside an async handler blocks the event loop exactly as much as a plain function would.
main thread (event loop)
│ pool.run(args)
▼
┌──────────── Piscina task queue ────────────┐
│ task task task … │
└─────┬──────────────┬──────────────┬────────┘
══════╪══════════════╪══════════════╪═══════ structured clone
▼ ▼ ▼ (args in, results out)
┌─────────┐ ┌─────────┐ ┌─────────┐
│ worker 1│ │ worker 2│ │ worker N│
│ isolate │ │ isolate │ │ isolate │
│ heap │ │ heap │ │ heap │
│ loop │ │ loop │ │ loop │
└─────────┘ └─────────┘ └─────────┘
┌─ main thread: V8 isolate + event loop ─────────────────────┐
│ handler: const r = await pool.run(args) │
└─────────────┬───────────────────────────────┴──────────────┘
│ enqueue task ▲ promise settles
▼ │
┌──────────── Piscina task queue ────────────┐│
│ task task task … ││
└─────┬──────────────┬──────────────┬────────┘│
═══════╪══════════════╪══════════════╪═════════╪═══ structured clone
▼ ▼ ▼ │ args in, results out
┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ worker 1│ │ worker 2│ │ worker N│ │
│ isolate │ │ isolate │ │ isolate │ │
│ heap │ │ heap │ │ heap │ │
│ evt loop│ │ evt loop│ │ evt loop│ │
└────┬────┘ └────┬────┘ └────┬────┘ │
└──────────────┴──────────────┴─────────┘
┌─ libuv thread pool (separate; UV_THREADPOOL_SIZE) ─────────┐
│ native C++ only: fs, dns.lookup, zlib, async crypto │
│ never runs your JavaScript │
└────────────────────────────────────────────────────────────┘
The main thread queues tasks in Piscina, idle workers with their own isolate, heap and event loop run them, and arguments and results are copied across the structured-clone boundary. libuv's separate pool runs only native work.
How does it work? The life of one request through the pool
Following a single request end to end makes the moving parts concrete:
- The handler calls
pool.run(task, options).taskis a plain value that can be structured-cloned.optionscan carry atransferListand anAbortSignal. The call returns a promise immediately, so the handler yields and the event loop goes back to serving other requests. - Piscina picks a worker. If a worker is idle, the task goes straight to it. If none is idle and the pool is below
maxThreads, Piscina starts another worker. Otherwise the task waits in the pool's queue, and if that queue has reachedmaxQueue,run()rejects instead. - The task crosses the thread boundary. The arguments are structured-cloned into the worker's isolate, and any buffers in
transferListare moved rather than copied. - The worker runs your exported function. It runs on its own OS thread with its own event loop, so a long synchronous loop there blocks only that worker. The function can be synchronous or return a promise.
- The result comes back. The return value is cloned back to the main thread and the
run()promise resolves. If the function throws or its promise rejects, therun()promise rejects with that error. - The worker goes back to idle. It stays alive with its modules already loaded, so the next task skips isolate startup. With
minThreadsbelowmaxThreads, Piscina can shut down workers that stay idle for longer than its idle timeout.
The main thread only does steps 1, 3 and 5: enqueueing, cloning arguments and cloning results. Everything expensive happens in step 4, on another core.
Setting up a Piscina pool for a CPU-heavy route
The structure has three parts. First, a worker module, such as resize.worker.js, that exports one function taking a plain object and returning a result. Second, a single Piscina instance created at process start, with filename pointing at that module. Third, a route handler that does await pool.run(task) where it used to call the heavy function directly. The handler stays async, and the event loop is free to accept other requests while a worker does the computation.
Create the pool once, at module scope, and export it. If you construct a pool inside a handler, every request spins up new threads and pays isolate startup cost again. That startup cost is the overhead pooling is meant to remove.
Keep payloads small
Arguments to pool.run() and the return values go through the structured clone algorithm. That means they are copied, not shared. Cloning a large object graph costs time on the main thread, which brings back the blocking you were trying to avoid. Some ways to keep payloads small:
- Pass references, not data. Send an ID or a storage key, and let the worker fetch what it needs.
- Transfer binary data. If you already hold an
ArrayBuffer, list it intransferList. Ownership moves to the worker without a copy, and the main thread's view of that buffer becomes unusable. - Share read-mostly data. Put large lookup tables or model weights in a
SharedArrayBufferthat every worker reads, instead of cloning them into each task.
Workers start empty
Each worker has its own isolate and its own module graph. Your database pool, Redis client, in-memory caches and config singletons do not exist inside the worker. If a worker imports them, it creates separate copies. Keep worker modules pure: inputs in, result out. Do any I/O in the main thread, before or after the call to pool.run().
// resize.worker.js: runs in a worker isolate. Keep it pure, with no DB clients, caches or singletons.
module.exports = ({ width, pixels }) => {
const src = new Uint8Array(pixels); // ArrayBuffer was transferred, not copied
let checksum = 0;
for (let i = 0; i < src.length; i++) checksum = (checksum + src[i]) >>> 0; // stand-in for real CPU work
return { width, bytes: src.length, checksum }; // small result, cheap to clone back
};
// pool.js: create ONE pool at process start and share it
const path = require('node:path');
const Piscina = require('piscina');
module.exports = new Piscina({ filename: path.resolve(__dirname, 'resize.worker.js') });
// server.js: the handler awaits the pool instead of calling the heavy function inline
const http = require('node:http');
const pool = require('./pool');
http.createServer(async (req, res) => {
const chunks = [];
for await (const chunk of req) chunks.push(chunk);
const buf = Buffer.concat(chunks);
// Copy into a standalone ArrayBuffer. Small Buffers can sit on a shared internal pool.
const pixels = buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength);
try {
// transferList moves ownership to the worker. `pixels` is detached here afterwards.
const result = await pool.run({ width: 800, pixels }, { transferList: [pixels] });
res.writeHead(200, { 'content-type': 'application/json' });
res.end(JSON.stringify(result));
} catch (err) {
res.writeHead(500);
res.end('worker failed');
}
}).listen(3000);
A pure worker module, a single module-scoped Piscina pool, and an HTTP handler that awaits pool.run() and transfers the request's binary payload to the worker with transferList instead of cloning it.
How many threads should a Piscina pool have?
Start from the CPUs the container can actually use, not the host's core count. Inside a container, os.cpus().length reports every core on the node. os.availableParallelism(), where your runtime provides it, respects CPU affinity but may not reflect a CFS quota. On cgroup v2, read cpu.max yourself: a value of quota and period such as 200000 100000 means two CPUs' worth of time, however many cores are visible. Piscina's defaults are derived from the CPU count it detects, so set minThreads and maxThreads explicitly.
Subtract headroom. The main thread still parses requests, serializes results and runs its own GC. With a quota of N CPUs, maxThreads = N - 1 (minimum 1) is a sound starting point. Set minThreads equal to maxThreads for steady traffic, because spawning a worker on demand adds isolate startup cost to the request that triggered it.
Two failure modes bound the number:
- Memory. Each worker is a separate V8 isolate with its own heap. Peak memory is roughly the main heap plus
maxThreadstimes the per-worker peak. If that exceeds the container limit, the kernel OOM-kills the whole process. Cap worker heaps withresourceLimits. - Oversubscription. More busy threads than quota exhausts the CFS budget early in each period. The process is then throttled until the period ends, which inflates p99 latency even while average CPU utilization looks moderate.
| Sizing input | Effect on pool |
|---|---|
CFS quota (cpu.max) |
Upper bound for maxThreads |
availableParallelism() |
Upper bound when no quota is set |
| Main-thread headroom | Subtract one from the bound |
| Container memory limit | Caps maxThreads by per-worker heap |
| Traffic shape | Steady traffic: minThreads = maxThreads |
'use strict';
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const Piscina = require('piscina');
const WORKER_HEAP_MB = 256; // old-gen cap per worker isolate
const WORKER_BUDGET_MB = 320; // heap cap plus young gen and native overhead
const MAIN_RESERVE_MB = 512; // main thread heap, buffers, runtime overhead
function readCgroup(file) {
try { return fs.readFileSync(`/sys/fs/cgroup/${file}`, 'utf8').trim(); }
catch { return null; } // not cgroup v2, or file not present
}
// CPU bound: the CFS quota wins over the visible core count.
const visible = typeof os.availableParallelism === 'function'
? os.availableParallelism() : os.cpus().length;
const [quota, period] = (readCgroup('cpu.max') || 'max').split(' ');
const cpuBound = quota === 'max'
? visible : Math.min(visible, Math.floor(Number(quota) / Number(period)));
// Memory bound: how many worker budgets fit beside the main thread.
const memMax = readCgroup('memory.max');
const memBound = !memMax || memMax === 'max' ? Infinity
: Math.floor((Number(memMax) / 2 ** 20 - MAIN_RESERVE_MB) / WORKER_BUDGET_MB);
// Leave one CPU for the main thread; never go below one worker.
const threads = Math.max(1, Math.min(cpuBound - 1, memBound));
const pool = new Piscina({
filename: path.resolve(__dirname, 'worker.js'),
minThreads: threads, // steady traffic: no on-demand isolate startup
maxThreads: threads,
resourceLimits: { maxOldGenerationSizeMb: WORKER_HEAP_MB },
});
console.log({ visible, cpuBound, memBound, threads });
module.exports = pool;
Sizes a Piscina pool from the cgroup v2 CPU quota and memory limit, falling back to os.availableParallelism(), reserves a CPU for the main thread, and caps each worker's heap with resourceLimits.
How do you handle overload, cancellation and timeouts in a worker pool?
Set maxQueue explicitly. Without it, a burst of requests piles up in memory and overload appears as rising latency. Clients time out and retry, and the queue grows further. Threads do not add capacity. They only spread a fixed CPU budget across cores, so a deep queue just delays the moment you say no. With a bound, pool.run() rejects once the queue is full. Catch that rejection and return 503 (or 429 if your API treats it as per-client throttling) with a Retry-After header. Callers then back off instead of hammering a saturated service.
Pass an AbortSignal to pool.run(). Tie it to the request's close event and to a deadline from AbortSignal.timeout(). A queued task that is aborted is dropped before it consumes a thread. A task already executing is a different case: synchronous JavaScript cannot be interrupted cooperatively, and Piscina stops a running task by terminating its worker. That is correct but expensive, because the replacement worker pays startup cost again. For long jobs, add cancellation checks inside the task by processing input in chunks and checking a shared flag between chunks, so a deadline can end the work cleanly.
A thrown error inside the task rejects the run() promise. Map it to a 500 and log it. If a worker crashes, Piscina replaces it, but the in-flight task fails, so callers must handle that rejection too. To contain slow memory leaks in native code or caches, cap heap with resourceLimits and periodically recycle workers. During graceful shutdown, stop accepting requests, let in-flight tasks drain, then call pool.destroy().
What should you monitor, and what does pooling buy you? An illustrative estimate
Export Piscina's own counters (queueSize, utilization, completed, and the waitTime and runTime histograms) next to event loop delay from perf_hooks.monitorEventLoopDelay. Read them together:
- High
waitTime, flatrunTime, utilization near 1: the pool is saturated. Add capacity or shed load. - Rising
runTime, low wait: the tasks themselves got slower, usually because of larger inputs or a regression. - Rising event loop delay while the pool is idle: something is still running inline on the main thread.
Debugging gets harder. Stack traces stop at the thread boundary, so log a task ID on both sides. APM and tracing context may not propagate into workers, so pass trace IDs explicitly in the task payload.
Illustrative estimate (assumed numbers, not a benchmark)
Assumptions:
- 4 cores, with 3 workers and 1 core left for the main thread.
- The resize task costs 20 ms of CPU.
- Structured-clone and dispatch overhead is about 0.3 ms per task, mostly paid on the main thread.
Formulas:
- Inline throughput ≈
1000 / task_ms - Pooled throughput ≈
workers × 1000 / (task_ms + overhead_ms)
| Scenario | Max throughput | Main-thread block per request | Effect on other routes |
|---|---|---|---|
| Inline, 20 ms task | 1000/20 = 50 req/s | 20 ms | Every request waits behind each resize |
| Pooled, 20 ms task | 3×1000/20.3 ≈ 148 req/s | ~0.3 ms | Event loop stays responsive |
| Inline, 0.1 ms hash | 1000/0.1 = 10,000 req/s | 0.1 ms | Negligible |
| Pooled, 0.1 ms hash | Bounded by main-thread overhead, ≈ 1000/0.3 ≈ 3,300 req/s | ~0.3 ms | Worse than inline |
For short tasks, the copying costs more than the work it offloads. You can batch many small tasks into one message, or keep them inline.
How can a Piscina pool save engineering time?
The time savings come from what you don't have to build or operate, not from the code being faster to write:
- No new service to stand up. The heavy function stays in the same repository, language, test suite, build and deploy pipeline. There is no new API contract to version, no service discovery, no extra on-call rotation, and no network failure modes between the API and the compute.
- A small, local refactor. The change is mechanical: move the CPU-heavy function into a worker module that takes a plain object and returns a plain result, create one pool at startup, and replace the inline call with
await pool.run(task). Callers above the handler don't change, because the handler was already async. - No hand-rolled pool. Writing a correct pool on raw
worker_threadsmeans building task queuing, worker reuse, replacement after crashes, back-pressure, cancellation, and per-task timing yourself. Piscina providesmaxQueue,AbortSignalsupport, worker replacement,resourceLimits, and thewaitTime/runTimehistograms, so that code doesn't have to be written, reviewed and maintained in-house. - Faster incident diagnosis. Once event loop delay and pool metrics sit on the same dashboard, the questions "is the pool saturated, did the task get slower, or is something still running inline?" can be answered by reading three graphs instead of profiling production. A slow heavy route also stops dragging down health checks and cheap routes, which removes a common source of confusing, cross-route latency alerts.
- A clear exit path. A pure worker module with a plain-object interface is already shaped like a job handler or a service endpoint. If the workload later outgrows one process, moving it to a job queue or a separate service starts from a well-defined boundary instead of untangling code that shared state with the request handler.
The savings shrink, or reverse, when the worker needs database clients, caches or other shared state. Each of those has to be duplicated per thread or redesigned, and debugging across threads is harder than debugging inline code. Keep the worker function pure and the pool stays cheap to own.
When should you avoid Piscina, and what are the alternatives?
If a task finishes in well under a millisecond, run it inline, or batch many items into one task so the dispatch cost is paid once. The rest of the decision depends on how long the work runs, how bursty it is, and whether the caller must wait for the result.
| Option | Fits when | Main cost |
|---|---|---|
| Inline on main thread | Task is shorter than dispatch overhead | Any growth in task size blocks the event loop |
| Piscina pool | Request-scoped CPU work of a few ms to a few seconds, spare cores on the host | Per-worker memory, separate module state |
| Job queue with workers | Long or bursty batch jobs; caller can poll or be notified | Queue infrastructure, retries, result storage |
| Separate service | Workload needs its own scaling, language or release cycle | Network hop, deployment, on-call surface |
| Native addon (Node-API) | Hot loop dominates and a compiled implementation is much faster | Build toolchain, crash risk inside the process |
| More processes (cluster or replicas) | All work is CPU-heavy, not one route | Higher baseline memory; one slow request still blocks its process |
Workers carry real development cost. Each worker is its own isolate, so database clients, caches and config loaded at module scope are duplicated per thread. Code inside a worker should stay pure: take inputs, return outputs, and hold no connections. Stack traces cross a thread boundary, debuggers attach per thread, and shutdown has to drain in-flight tasks before the process exits.
A bounded pool is still often the cheaper option. If one process has idle cores and the slow path is a single route, a pool with a queue limit and timeouts fixes event loop blocking without adding a new service, a new deploy pipeline or a network contract. Move to a queue or a separate service when the work outlives the request or needs to scale independently of the API.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to node.js worker threads piscina, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.