Node.js Worker Threads for CPU-Bound APIs: Piscina and Pool Sizing

How do you know event loop blocking is hurting your Node.js API?

This guide is for backend and platform engineers running latency-sensitive APIs that mix fast I/O-bound endpoints with occasional heavy computation.

Node.js runs all JavaScript for a process on one event loop thread. If a handler spends 200 ms hashing a password or rendering a PDF, every other request waits for that time to pass. That includes health checks and cheap cache hits. Throughput may look fine, but tail latency climbs for every concurrent request, and slow work on one endpoint shows up as unexplained p99 spikes on unrelated endpoints.

Confirm it with measurements. perf_hooks.monitorEventLoopDelay() returns a histogram you can sample with percentile(99) and max (values are in nanoseconds). Export those values next to request latency. A sustained high p99 loop delay that tracks your latency spikes shows the loop itself is stalled. To find the cause, capture a CPU profile with node --cpu-prof or the inspector under realistic load, then look for wide synchronous frames in your own code or in libraries.

If loop delay stays low while latency is high, the bottleneck is I/O, and threads will not fix it.

Copy// Minimal server showing how one sync CPU-heavy handler stalls the event loop.
// Run: node server.js, then hit /slow while polling /fast and watch the log.
const http = require('node:http');
const crypto = require('node:crypto');
const { monitorEventLoopDelay } = require('node:perf_hooks');

// Histogram of event loop delay, sampled every 20 ms. Values are in nanoseconds.
const loopDelay = monitorEventLoopDelay({ resolution: 20 });
loopDelay.enable();

// Every 5 s, report p50/p99/max loop delay in ms, then reset the window.
// In production, export these as metrics next to request latency instead of logging.
setInterval(() => {
  const ms = (ns) => (ns / 1e6).toFixed(1);
  console.log(
    `loop delay p50=${ms(loopDelay.percentile(50))}ms ` +
    `p99=${ms(loopDelay.percentile(99))}ms max=${ms(loopDelay.max)}ms`
  );
  loopDelay.reset();
}, 5000).unref();

http.createServer((req, res) => {
  if (req.url === '/slow') {
    // Synchronous key derivation: blocks the only JS thread until it finishes.
    // Every concurrent request, including /fast and health checks, waits here.
    const key = crypto.pbkdf2Sync('placeholder-password', 'placeholder-salt', 600000, 64, 'sha512');
    res.end(key.toString('hex').slice(0, 16));
    return;
  }
  // Cheap endpoint: should be sub-millisecond, but its p99 tracks /slow.
  res.end('ok');
}).listen(3000);

// To find which code is blocking, profile under load:
//   node --cpu-prof server.js
// Then open the generated .cpuprofile in Chrome DevTools and look for wide sync frames.

A small HTTP server that reports event loop delay percentiles from perf_hooks.monitorEventLoopDelay, so the p99 delay rises with a synchronous pbkdf2Sync handler that also slows the cheap /fast endpoint.

What are Node.js worker threads and how do they work?

A worker thread is a separate V8 isolate with its own heap and its own event loop, running on an OS thread inside the same process. Workers do not share JavaScript objects with the main thread. They communicate through postMessage, which copies data using the structured clone algorithm. That copy is the hidden cost: cloning a large JSON-like object means walking it on the sending thread and rebuilding it on the receiving thread. A big enough payload blocks the main loop during serialization, which is the exact stall you were trying to avoid.

There are two ways to avoid the copy for binary data:

• Transferable ArrayBuffers move ownership to the other thread at near-zero cost. The sender's buffer becomes detached, which suits one-way handoffs such as image bytes or compressed input.
• SharedArrayBuffer gives both threads the same memory, coordinated with Atomics. It fits long-lived shared state, but you take on the synchronization yourself.

Do not confuse worker threads with libuv's threadpool. That is a small, fixed pool of native threads, four by default and sized with UV_THREADPOOL_SIZE. Node uses it internally for fs, dns.lookup, async zlib, and some crypto calls such as pbkdf2 and scrypt. Your JavaScript never runs there.

Copy            one process, N CPU cores (shared by every thread below)
 ┌──────────────────────────────────────────────────────────────────┐
 │ main thread ── submit ──▶ Piscina task queue ──▶ worker 1 (V8)   │
 │ (event loop)  ◀── result ──────────────────────  worker 2 (V8)   │
 │      │                                           worker n (V8)   │
 │      └── fs / zlib / crypto ──▶ libuv threadpool (default 4)     │
 └──────────────────────────────────────────────────────────────────┘

All of these threads compete for the same cores. Offloading work keeps the event loop responsive, but it does not add CPU capacity.

Copy┌───────────────────────── one Node.js process ──────────────────────────┐
│                                                                        │
│  ┌──────────────┐                ┌────────── Piscina pool ──────────┐  │
│  │ main thread  │                │ ┌──────────┐     ┌────────────┐  │  │
│  │ V8 isolate   │──submit───────▶│ │task queue│────▶│ worker 1   │  │  │
│  │ event loop   │◀──result───────│ │  (FIFO)  │────▶│ worker 2   │  │  │
│  └──────┬───────┘                │ │          │────▶│ worker n   │  │  │
│         │                        │ └──────────┘     │ own V8 heap│  │  │
│         │                        │                  │ + loop each│  │  │
│         │                        │                  └────────────┘  │  │
│         │                        └──────────────────────────────────┘  │
│         │ fs, dns.lookup, zlib, some crypto (pbkdf2, scrypt)           │
│         ▼                                                              │
│  ┌──────── libuv threadpool (UV_THREADPOOL_SIZE, default 4) ────────┐  │
│  │ native threads; your JavaScript never runs here                  │  │
│  └──────────────────────────────────────────────────────────────────┘  │
│                                                                        │
│  ┌────────────────── shared by every thread above ──────────────────┐  │
│  │ core 0    core 1    core 2    ...    core N-1                    │  │
│  │ offloading protects the event loop; it adds no capacity          │  │
│  └──────────────────────────────────────────────────────────────────┘  │
│                                                                        │
└────────────────────────────────────────────────────────────────────────┘

The main thread submits tasks to Piscina's queue, and the queue dispatches them to workers that each have their own V8 isolate and event loop; built-in fs, dns.lookup, zlib and some crypto calls go to libuv's threadpool instead, and every one of these threads competes for the same CPU cores.

Why use a Piscina thread pool instead of a worker per request?

Spawning a Worker for each request looks simple, but every worker is a full V8 isolate with its own heap, its own event loop, and its own copy of any modules it imports. Startup cost is paid on every request, and for short tasks it can exceed the work itself. Memory scales with concurrency, and nothing caps that concurrency: a traffic spike creates as many isolates as there are in-flight requests, oversubscribing the cores and pushing the process toward its memory limit.

A pool fixes all three. Piscina keeps a set of long-lived workers, queues incoming tasks when every worker is busy, and hands each task to the next free thread. Isolate startup and module loading happen once per worker, not once per request.

The basic shape has three parts. A worker module exports a function that takes the task payload and returns a result or a promise. The main thread constructs one Piscina instance at startup, pointing filename at that module. Request handlers call await pool.run(payload) and send the resolved value as the response.

The development cost is real. Workers need separate entry files, and bundlers and TypeScript builds must emit them as standalone outputs at paths the pool can resolve at runtime. Module-level state such as caches, connections, and config is per thread and not shared with the main thread. Debugging spans multiple isolates, so stack traces and breakpoints are harder to follow. Piscina does not remove these costs, but it does handle scheduling, queuing, and worker replacement, which you would otherwise have to build, test, and maintain in a custom pool.

Copy// worker.js: runs inside each pooled thread. Module-level state here is per thread.
module.exports = function countPrimes({ limit }) {
  let count = 0;
  for (let n = 2; n <= limit; n++) {
    let prime = true;
    for (let d = 2; d * d <= n; d++) {
      if (n % d === 0) { prime = false; break; }
    }
    if (prime) count++;
  }
  return { limit, count }; // returned value is structured-cloned back to the main thread
};

// server.js: main thread. Build one pool at startup, not one per request.
const http = require('node:http');
const path = require('node:path');
const os = require('node:os');
const Piscina = require('piscina');

const pool = new Piscina({
  filename: path.resolve(__dirname, 'worker.js'), // must resolve to the built worker file at runtime
  maxThreads: os.availableParallelism(),          // bound concurrency to the cores we have
});

http.createServer(async (req, res) => {
  const url = new URL(req.url, '');
  const limit = Math.min(Number(url.searchParams.get('limit')) || 100000, 5000000);
  try {
    // If every worker is busy, the task waits in Piscina's queue instead of spawning a new isolate.
    const result = await pool.run({ limit });
    res.writeHead(200, { 'content-type': 'application/json' });
    res.end(JSON.stringify(result));
  } catch (err) {
    res.writeHead(500, { 'content-type': 'text/plain' });
    res.end('worker task failed');
  }
}).listen(3000);

A CPU-bound worker module and an HTTP server that creates a single Piscina pool at startup and sends each request's work to it with pool.run().

How many threads should a Piscina pool have?

For pure CPU work, a thread only helps if a CPU is free to run it. The useful pool size is roughly the number of CPUs the process can actually use, minus one for the main thread, which still parses requests, serializes responses and runs libuv callbacks.

The trap is where that CPU number comes from. os.cpus() can report the host's cores, not the container's CPU quota. os.availableParallelism(), where your runtime has it, is a better input but may still not reflect a CFS quota. Set the size through explicit configuration, such as an environment variable, and check it against the container limit at startup.

Horizontal scaling multiplies the problem. Every process and every pod runs its own pool, so the real thread count is processes × threads per pool.

Illustrative estimate. Assume a 32-core host, a container limited to 4 CPUs, and roughly 30 MB of heap per worker isolate.

In the second row, 128 runnable threads share 4 CPUs of quota. They use up the quota early in each scheduling period, and then the kernel throttles the whole container until the next period begins. Throughput stays flat while tail latency rises, because work stalls on throttling rather than waiting in a queue you can see. Each extra worker is also a separate V8 isolate with its own heap. In a tight memory limit, oversizing the pool can trigger out-of-memory kills before CPU saturation becomes the visible problem.

Copy// pool.js: size the Piscina pool from explicit config, checked against the container CPU limit
const os = require('node:os');
const fs = require('node:fs');
const path = require('node:path');
const Piscina = require('piscina');

// Read the CFS quota from cgroup v2 ("max 100000" means no limit).
function cgroupCpuLimit() {
  try {
    const [quota, period] = fs.readFileSync('/sys/fs/cgroup/cpu.max', 'utf8').trim().split(' ');
    if (quota === 'max') return null;
    return Math.max(1, Math.floor(Number(quota) / Number(period)));
  } catch {
    return null; // not cgroup v2, or not in a container
  }
}

// May report host cores. Use it only as a fallback.
const runtimeCpus = typeof os.availableParallelism === 'function'
  ? os.availableParallelism()
  : os.cpus().length;

const cpus = cgroupCpuLimit() ?? runtimeCpus;
// Leave one CPU for the main thread (HTTP parsing, serialization, libuv callbacks).
const ceiling = Math.max(1, cpus - 1);

// Explicit config wins. Compute the default only when the variable is unset.
const configured = Number.parseInt(process.env.POOL_THREADS ?? '', 10);
const threads = Number.isInteger(configured) && configured > 0 ? configured : ceiling;

if (threads > ceiling) {
  // Remember: total threads = processes x pods x this number.
  console.warn(`POOL_THREADS=${threads} exceeds CPU limit ${cpus} - 1; expect CFS throttling`);
}

const pool = new Piscina({
  filename: path.resolve(__dirname, 'worker.js'),
  minThreads: threads,
  maxThreads: threads,
});

module.exports = { pool };

Sizes a Piscina pool from an explicit POOL_THREADS setting, falls back to the cgroup v2 CPU quota minus one, and warns at startup if the configured size would oversubscribe the container.

How do you add backpressure and timeouts so a full pool fails fast?

An unbounded queue hides overload. When requests arrive faster than the pool can finish them, nothing fails at first. Tasks pile up, each one holding its payload in memory, and every new request waits behind all of them. By the time anyone notices, clients have timed out and retried, which adds more work to a queue that is already behind.

The fix is to cap the queue so overload shows up as an explicit error. Piscina's maxQueue option limits how many tasks can wait. Once the queue is full, run() rejects instead of accepting more work. Don't rely on catching that rejection after the fact. Before you submit, compare piscina.queueSize against your limit. If the queue is full, respond immediately with 503, or with 429 if the limit is per client, and include a Retry-After header so well-behaved clients back off.

To choose the limit, work backward from your latency budget:

Illustrative estimate (assumed numbers, not a benchmark). Assume 4 threads and 50 ms per task. Worst-case queue wait ≈ queue length × task time ÷ threads. With a 500 ms budget: 500 × 4 ÷ 50 = a queue of about 40 tasks.

For per-task deadlines, pass an AbortSignal in the run() options. AbortSignal.timeout() works well for this. If the signal fires while the task is still queued, Piscina drops it. If the task is already running, Piscina terminates that worker and starts a replacement. Aborting a running task therefore costs a thread restart, so set timeouts as a safety net rather than as normal flow control.

Rejecting requests early is less pleasant for clients. In exchange, the requests you accept finish within a predictable time.

Running worker pools in production: memory limits, crashes, shutdown and metrics

Memory limits

Pass resourceLimits (for example maxOldGenerationSizeMb) when you create the pool to cap each worker's V8 heap. A worker that exceeds its cap is terminated with an out-of-memory error. The pending task's promise rejects, and the pool replaces the worker. A thrown exception inside a task also rejects the caller's promise. Map both to a 5xx response and log the error code so you can tell crashes apart from ordinary failures.

These caps cover only the JavaScript heap. Workers share the process with the main thread, so a segfault in a native addon, an abort(), or growth in off-heap memory such as Buffers or native allocations will take down the whole service. Isolation in a thread pool is partial.

Shutdown

On SIGTERM, stop intake first: fail readiness checks and return 503 for new CPU work. Then let queued tasks drain within a deadline shorter than your orchestrator's kill timeout. Reject whatever remains with a clear error, and close the pool.

Metrics

Export these:

• Queue depth.
• Task wait time and run time, as separate histograms.
• Pool utilization.
• Rejected and timed-out task counts.
• Main-thread event loop delay (perf_hooks.monitorEventLoopDelay).

Reading wait time against run time tells you what kind of problem you have. Rising wait time with flat run time means the pool is saturated. Rising run time points to slower tasks or CPU contention. When utilization stays low while event loop delay is high, something is still blocking the main thread. These signals answer the sizing question from dashboards, before an incident forces you to.

Measure before and after you offload any task. Arguments and results are copied between threads, so for small tasks the serialization and queueing overhead can cost more than running the work inline.

When should you avoid worker threads, and what are the alternatives?

In-request worker threads fit when four conditions hold. The task is CPU-bound, such as hashing, image resizing, or parsing a large document. Its cost is bounded and predictable, usually tens to hundreds of milliseconds. The caller can tolerate that latency. Inputs and outputs are small, or can be passed as transferable buffers.

If any of those fails, look elsewhere:

• Tiny tasks. When copying arguments costs about as much as computing on them, offloading adds latency without freeing the event loop.
• I/O-bound work. The event loop already handles database calls and HTTP well. Threads add memory and add nothing for waiting.
• Long-running or bursty batch jobs. These occupy pool slots for seconds or minutes and starve interactive requests. They also lose progress on restart.
• Strong fault isolation. Workers share one process. A native crash or memory exhaustion can take down the whole server.

Try caching or eliminating the work before anything else. The cheapest CPU cycle is the one you never spend.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to node.js worker threads, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.