Node.js Worker Threads for CPU-Bound APIs: Pools and Message Costs

Is your Node.js API actually CPU-bound?

A Node.js process runs your JavaScript on a single thread: the event loop. I/O-bound work, such as database queries, HTTP calls, and file reads, spends most of its time waiting. That waiting happens outside JavaScript, so the loop moves on and serves other requests in the meantime. CPU-bound work keeps the thread busy running your code. Examples include hashing, image resizing, parsing a large JSON payload, PDF generation, and compression. While that code runs, nothing else on the loop can run.

The symptom is collateral latency. A health check or a cached lookup that normally returns in a couple of milliseconds suddenly takes hundreds, because it is queued behind one request that is computing. Under load this appears as p99 spikes that do not correlate with any slow dependency.

async/await does not fix this. Marking a function async only changes how its result is delivered. The synchronous loop inside it still runs to completion on the same thread before the loop can do anything else. Wrapping a CPU-heavy function in a Promise just moves where the blocking happens.

Measure the problem before you change your architecture. monitorEventLoopDelay from perf_hooks returns a histogram. Once you call enable(), it samples how late the loop's timers fire. It reports mean, max, and percentile() values in nanoseconds. If p99 delay climbs whenever a specific endpoint is hit, you have a CPU-bound problem. If delay stays flat while latency rises, look at I/O instead. Threads will not help in that case.

Copyconst http = require('node:http');
const crypto = require('node:crypto');
const { monitorEventLoopDelay } = require('node:perf_hooks');

// Sample event-loop delay every 10 ms. Histogram values are in nanoseconds.
const loopDelay = monitorEventLoopDelay({ resolution: 10 });
loopDelay.enable();

const ms = (ns) => (ns / 1e6).toFixed(1);

// Log, then reset every 5 s so each line covers only the latest window.
setInterval(() => {
  console.log(
    `loop delay ms: mean=${ms(loopDelay.mean)} ` +
    `p99=${ms(loopDelay.percentile(99))} max=${ms(loopDelay.max)}`
  );
  loopDelay.reset();
}, 5000).unref();

const server = http.createServer(async (req, res) => {
  if (req.url === '/hash') {
    // The handler is async, but pbkdf2Sync is synchronous.
    // It holds the only JS thread until the hash finishes.
    const key = crypto.pbkdf2Sync(
      'placeholder-password', 'placeholder-salt', 500000, 64, 'sha512'
    );
    res.end(key.toString('hex'));
    return;
  }
  // Cheap endpoint. It should return almost instantly,
  // but it waits behind any /hash request that is computing.
  res.end('ok');
});

// Try it: poll /health in a loop, then hit /hash once.
// Watch p99 and max jump in the log while /health slows down.
server.listen(3000);

A minimal HTTP server that logs event-loop delay percentiles from perf_hooks.monitorEventLoopDelay, showing that a synchronous hash inside an async handler blocks an unrelated cheap endpoint and drives up p99 and max loop delay.

What node.js worker threads provide, and how they differ from child_process and cluster

Worker threads only help the first case, so it is worth knowing exactly what they give you.

The worker_threads module runs JavaScript on additional OS threads inside the same process. Each worker gets its own V8 isolate, its own event loop, and its own heap. Workers share no JavaScript objects with the main thread. They communicate through MessagePort channels, and by default every message is copied.

That isolation is what makes workers expensive. Each new worker has to bootstrap a fresh Node.js environment, load its modules, and warm up its JIT before it does useful work. It then holds that heap for as long as it lives. If you spawn a worker per request, you pay this startup cost on the hot path, and memory grows without limit under load. The fix is a small set of long-lived workers fed by a bounded queue:

Main thread (HTTP handlers) → run(task) → Bounded task queue → dispatch to idle worker → Worker 1 / Worker 2 / … / Worker N (each with its own MessagePort) → result via postMessage → resolve the caller's promise

Comparison of the three options

cluster spreads incoming requests across processes. It does not offload a slow computation that runs inside a single request. Worker threads do.

Copy+--------------------------------------------------------------------+
| Node.js process (one PID)                                          |
|                                                                    |
|  +------------------------------+     +-------------------------+  |
|  | Main thread: isolate + loop  |     | Bounded task queue      |  |
|  |  HTTP handlers               |---->|  [t5][t4][t3] (capped)  |  |
|  |  pool.run(task) -> Promise   |     |  full -> reject / 503   |  |
|  +------------------------------+     +-------------------------+  |
|          ^                                         |               |
|          | resolve caller's promise                | dispatch to   |
|          |                                         v idle worker   |
|  +-------+-----------------------------------------+------------+  |
|  |            Worker pool (long-lived, fixed size N)            |  |
|  |                                                              |  |
|  |  +------------+   +------------+         +------------+      |  |
|  |  | Worker 1   |   | Worker 2   |         | Worker N   |      |  |
|  |  | own isolate|   | own isolate|   ...   | own isolate|      |  |
|  |  | own loop   |   | own loop   |         | own loop   |      |  |
|  |  | own heap   |   | own heap   |         | own heap   |      |  |
|  |  +-----+------+   +-----+------+         +-----+------+      |  |
|  |        |                |                      |             |  |
|  |   MessagePort      MessagePort            MessagePort        |  |
|  |   postMessage      postMessage            postMessage        |  |
|  +--------------------------------------------------------------+  |
|                                                                    |
+--------------------------------------------------------------------+

One process: the main thread enqueues tasks into a bounded queue, a fixed pool of long-lived workers (each with its own isolate, event loop and heap) takes them, and results return over MessagePort to resolve the caller's promise.

How to build a minimal worker pool, then switch to Piscina

A hand-written pool

Start with a fixed array of Worker instances, sized to the cores you intend to dedicate, and a plain array used as a FIFO queue. Each queued entry holds the task payload plus the resolve and reject functions from a new Promise(...). That promise is what callers await. When a worker is idle, shift the next entry, record it as that worker's in-flight task, and postMessage the payload. Then wire three listeners per worker:

• message resolves the in-flight promise and pulls the next task.
• error rejects it, so an exception thrown inside the worker reaches the caller instead of disappearing.
• exit with a non-zero code rejects any in-flight task and spawns a replacement. Without this step, a crashed worker silently shrinks the pool.

That is roughly a hundred lines, and it still lacks a queue bound, cancellation, and scaling down.

Switching to Piscina

Piscina implements the same model and adds the production features. Configure it with filename, minThreads/maxThreads, maxQueue, and idleTimeout:

• maxQueue makes run() reject immediately once the backlog is full, instead of letting memory and latency grow.
• idleTimeout retires surplus threads after a quiet period.
• Passing { signal } to run() removes a queued task when its AbortSignal fires. A task that has already started is not stopped this way.

In a Fastify or Express handler, create an AbortController, abort it when the client disconnects, and await pool.run(payload, { signal }). Map a queue-full rejection to a 503. You get one dependency to upgrade instead of concurrency code your team has to maintain.

Copyimport os from 'node:os';
import Fastify from 'fastify';
import { Piscina } from 'piscina';

const MAX_QUEUE = 256; // backlog bound: fail fast instead of growing latency

const pool = new Piscina({
  filename: new URL('./hash-worker.js', import.meta.url).href,
  minThreads: 2,
  maxThreads: Math.max(2, os.availableParallelism() - 1), // leave a core for the event loop
  maxQueue: MAX_QUEUE,
  idleTimeout: 30_000, // retire surplus threads after 30 s of quiet
});

const app = Fastify();

app.post('/hash', async (request, reply) => {
  // Shed load before enqueueing. Do not rely on the text of the rejection.
  if (pool.queueSize >= MAX_QUEUE) {
    return reply.code(503).send({ error: 'busy, retry later' });
  }

  // If the client goes away, drop the task while it is still queued.
  // A task that has already started keeps running.
  const controller = new AbortController();
  request.raw.on('close', () => controller.abort());

  try {
    const digest = await pool.run(request.body, { signal: controller.signal });
    return { digest };
  } catch (err) {
    if (err.name === 'AbortError') return reply.code(499).send(); // client gone
    throw err; // a worker exception reaches Fastify's error handler as a 500
  }
});

await app.listen({ port: 3000 });

A Fastify handler awaits a Piscina task. The pool is configured with maxQueue and idleTimeout, sheds load with a 503 when the queue is full, and uses an AbortSignal to drop the queued task when the client disconnects.

The hidden cost of postMessage: structured clone and when offloading loses

postMessage does not share memory. It runs the structured clone algorithm, which serializes the payload on the sending thread and deserializes it on the receiving thread. A round trip does this twice: the request goes in, and the result comes back. The main thread pays to serialize the input and to deserialize the output. Offloading exists to free the main thread, so if those two steps take longer than the task itself, you have made things worse.

Illustrative estimate (assumed numbers, not a benchmark)

Assume a 2 MB input object and a 0.5 MB result, and assume clone throughput of 500 MB/s for each serialize or deserialize step (0.5 MB/ms).

• Time for one clone step: clone_ms ≈ size_MB / throughput_MB_per_ms
• Main-thread cost: clone(in) + clone(out) = 4 ms + 1 ms = 5 ms
• Added end-to-end latency: about 2 × (clone(in) + clone(out)) = 10 ms, plus fixed handoff and queueing time, which this estimate does not model

Large inputs are not the only losing case. A task that takes microseconds on a 1 KB input also loses, because fixed per-message overhead outweighs the work.

Clone cost depends on the shape of the data, not only its byte size. Deeply nested objects with many small properties clone more slowly than flat buffers of the same size. Do not reuse these numbers. Time structuredClone on your real payloads, measure your real compute time, and put both into the formula above.

Reducing message overhead: transfers, SharedArrayBuffer, compact inputs, and batching

Once you know which side of that ledger you are on, four techniques shrink the message side.

Transfer instead of copy. Passing an ArrayBuffer in the transfer list of postMessage moves ownership to the receiving thread instead of cloning the bytes. The catch is that the sender's buffer is detached afterwards. Its byteLength becomes zero, and any typed array view over it reads as empty. A common bug is a request handler that transfers a buffer and later logs or hashes it on the main thread. Treat a transferred buffer as consumed, and copy it first if you still need it.

Share memory. A SharedArrayBuffer is visible to both threads with no copy and no transfer, which suits counters, progress flags, or large read-mostly tables. You also take on data races. Plain reads and writes can interleave unpredictably, so you need Atomics operations and, for anything beyond single-word updates, lock-like coordination that is hard to test.

Send compact inputs. Pass an ID, a file path, or a flat buffer, and let the worker load or decode the data itself. A large object graph pays the clone cost on every call.

Batch. If tasks take microseconds each, group many into one message. This spreads the fixed per-message overhead across the batch, at the cost of higher latency for the first item.

Copyconst { Worker, isMainThread, parentPort, workerData } = require('node:worker_threads');

if (isMainThread) {
  // Shared counter: one Int32 slot visible to both threads, never copied.
  const progress = new Int32Array(new SharedArrayBuffer(4));
  const worker = new Worker(__filename, { workerData: { progress } });

  // Transfer: ownership of buf moves to the worker, so no bytes are cloned.
  const buf = new Float64Array(1_000_000).fill(1).buffer;
  worker.postMessage({ type: 'sum', buf }, [buf]);
  console.log('after transfer, byteLength =', buf.byteLength); // 0: detached
  // Bug pattern: hashing or logging buf here sees nothing. Use buf.slice(0) first if needed.

  // Batch: many tiny tasks share one message's fixed overhead.
  worker.postMessage({ type: 'batch', items: [3, 5, 7, 11] });

  worker.on('message', (msg) => {
    console.log(msg, 'progress =', Atomics.load(progress, 0));
    if (msg.type === 'batch') worker.terminate(); // messages arrive in order
  });
} else {
  const { progress } = workerData; // same memory as the main thread's view
  parentPort.on('message', (msg) => {
    if (msg.type === 'sum') {
      const view = new Float64Array(msg.buf); // worker now owns these bytes
      let total = 0;
      for (const x of view) total += x;
      Atomics.add(progress, 0, 1); // atomic; a plain progress[0]++ could race
      parentPort.postMessage({ type: 'sum', total });
    } else if (msg.type === 'batch') {
      const results = msg.items.map((n) => n * n);
      Atomics.add(progress, 0, msg.items.length);
      parentPort.postMessage({ type: 'batch', results });
    }
  });
}

A single-file worker_threads example that transfers an ArrayBuffer and shows that it is detached on the sender, updates a SharedArrayBuffer counter with Atomics, and batches small tasks into one message.

Sizing the pool, backpressure, and failure handling

Start from os.availableParallelism() (or os.cpus().length on older runtimes) and subtract. The main thread still parses requests and serializes responses, and the libuv threadpool (four threads by default, set by UV_THREADPOOL_SIZE) runs crypto, zlib, and filesystem work. On an 8-core host, a pool of 6 is a better starting point than 8.

In containers, the core count can mislead. It may report the host's CPUs while a cgroup quota allows the process only two cores of time. Ten busy workers on a two-core quota get throttled, which shows up as latency spikes rather than errors. Size the pool to the quota. Each worker also has its own V8 heap, so memory grows with pool size. The resourceLimits option on Worker caps each heap and turns a runaway worker into a worker failure instead of a container OOM kill.

Admission control

An unbounded queue keeps the API up while latency grows without limit, and clients often time out on work that still runs. Cap the queue (Piscina exposes maxQueue) and return 503 with Retry-After when it is full. Rejecting early costs less than finishing late.

Failures

Give every task a timeout and cancel it with an AbortSignal. When a worker crashes, reject the request with an error that identifies the task, then replace the worker. Pools such as Piscina replace failed workers for you, but your handler still has to map the rejection to a 5xx response.

Verifying

Load test while recording event-loop delay with perf_hooks.monitorEventLoopDelay, queue depth, and queue wait time versus run time for each task. If wait time dominates, the pool is too small. If run time grows as you add workers, the workers are competing for CPU.

When to avoid worker threads, alternatives, and impact on cost and engineering time

Worker threads are the wrong tool when each task is small or I/O-bound. If a task runs for less time than it takes to clone its input and output, offloading adds latency and frees no CPU. Avoid them when code depends on things that cannot cross a thread boundary:

• Shared module state. Each worker has its own isolate and module cache, so caches and singletons are duplicated per worker, not shared.
• Open connections and closures. Sockets, database clients, and functions cannot be structured-cloned.
• Native addons. An addon that is not context-aware either fails to load in a worker or is unsafe to use there.

There are also real development costs. You have to restructure code around plain serializable inputs and outputs. Each worker carries its own heap, so memory grows with pool size. You also take on pool metrics, timeouts, and crash handling. A separate job service costs more to set up, but it isolates failures and scales on its own.

Piscina removes most of the pool code itself: queueing, worker replacement, cancellation, and resource limits. That code is easy to get subtly wrong when you write it yourself. The bigger saving comes earlier. Profile the workload before writing any worker code, then choose from the table above. That avoids a migration you would later have to undo.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to node.js worker threads, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a route's URL or method automatically notifies the services that consume it — no hunting through repos for callers.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Consumers learn about a breaking change at review time rather than from production errors.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.