How do you know your Node.js API is CPU-bound and blocking the event loop?

A request is I/O-bound when most of its time is spent waiting on a database, network call or disk. Node hands that wait to the OS or libuv and keeps serving other requests. A request is CPU-bound when its time goes to computation in JavaScript, such as image resizing, synchronous hashing, large JSON.parse calls or compression. That computation runs on the single main thread. While it runs, no other callback can execute, so every in-flight request on the process stalls, including health checks.

Confirm this before adding threads. Start by measuring event loop delay with perf_hooks.monitorEventLoopDelay(). It returns a histogram that reports values in nanoseconds. Healthy I/O-bound services usually show p99 delay of a few milliseconds. If p99 delay tracks your slow endpoints, you have blocking work. Next, capture a CPU profile with node --cpu-prof under realistic load to find the specific functions consuming the main thread. If the profile is dominated by idle time or GC rather than your code, worker threads are the wrong fix.

Illustrative estimate

Assumed inputs:

  • Each request needs 50 ms of CPU.
  • 10 requests arrive at once.
  • At least 4 cores are free.
  • Message-passing overhead is ignored.
Setup Max throughput Last request done Mean completion
Main thread 20 req/s (1 ÷ 0.05 s) 500 ms 275 ms
4-worker pool 80 req/s 150 ms (batches of 4, 4, 2) 90 ms

The bigger difference is that with the pool, the main thread stays free for cheap requests throughout.

// Run with: node --cpu-prof server.js  (writes a .cpuprofile on clean exit)
const http = require('node:http');
const { monitorEventLoopDelay } = require('node:perf_hooks');

// Sample event loop delay every 20 ms. The histogram reports nanoseconds.
const loopDelay = monitorEventLoopDelay({ resolution: 20 });
loopDelay.enable();

const ms = (ns) => (ns / 1e6).toFixed(1);

// Log p50/p99/max every 5 s, then reset so each window stands alone.
setInterval(() => {
  console.log(
    `loop delay ms p50=${ms(loopDelay.percentile(50))} ` +
    `p99=${ms(loopDelay.percentile(99))} max=${ms(loopDelay.max)}`
  );
  loopDelay.reset();
}, 5000).unref();

// Simulates ~50 ms of synchronous CPU work on the main thread.
function burnCpu(durationMs) {
  const end = Date.now() + durationMs;
  let x = 0;
  while (Date.now() < end) x += Math.sqrt(x + 1);
  return x;
}

http.createServer((req, res) => {
  if (req.url === '/slow') {
    burnCpu(50); // blocks every other request, including /health
    res.end('done\n');
  } else {
    res.end('ok\n'); // cheap endpoint: should be fast unless the loop is blocked
  }
}).listen(3000);

// Stop with Ctrl+C so --cpu-prof can flush the profile.
process.on('SIGINT', () => process.exit(0));

A minimal HTTP server that logs event loop delay percentiles with perf_hooks.monitorEventLoopDelay so you can see p99 delay rise when a 50 ms synchronous endpoint is under load, and run it with --cpu-prof to locate the blocking function.

Who should use worker threads?

Worker threads fit a specific profile. Use them if most of the following are true:

  • You run a Node.js API or service that mixes cheap requests with expensive computation. Typical examples are image or PDF processing, synchronous hashing or signing, parsing or validating large payloads, template rendering of big documents, compression done in JavaScript, and data transformation or scoring logic written in JS.
  • Profiling shows your own JavaScript on the main thread is the bottleneck, not I/O waits, GC or the libuv threadpool.
  • The expensive work is a function with plain-data inputs and outputs. If you can describe the task as "take this buffer or object, return that result", it can move to a worker with little restructuring.
  • The work is too frequent or too latency-sensitive to push to a separate service or job queue, but heavy enough that one task blocking the loop hurts other requests.
  • You want to keep one deployable. The team already owns a Node service and would rather not introduce a new language, a new service boundary or a message broker just to move one hot path.

Worker threads are a poor fit for teams whose services are I/O-bound, whose CPU work already runs through async crypto, zlib or fs on the libuv threadpool, or whose tasks finish in well under a millisecond. The last section of this article covers those cases and the alternatives.

What are Node.js worker threads and how do they work?

The worker_threads module runs JavaScript on additional OS threads inside the same process. Each Worker gets its own V8 isolate, its own event loop and its own heap. Nothing is shared by default. That separation is why a tight loop in a worker cannot stall your HTTP handlers. It also means the two sides communicate only through postMessage() on a MessagePort, either the implicit one between parent and worker or explicit ones from a MessageChannel.

Isolation has a cost. Starting a worker boots a fresh isolate, runs Node's bootstrap and loads your module graph again. Each worker carries its own heap and its own copies of the modules it imports. That startup cost usually exceeds the few milliseconds of work you want to offload, so spawning a worker per request makes latency worse and lets memory grow with concurrency. Create workers once, keep them alive, and feed them tasks through a queue.

Messages are copied with the structured clone algorithm, which limits what can cross the boundary:

  • Functions throw a DataCloneError.
  • Class instances arrive as plain objects. The prototype and its methods are lost.
  • Sockets, request/response objects and many native handles cannot be posted. Send the data the worker needs, not the object that holds it.
                 main thread (event loop: HTTP, I/O callbacks)
                         │ submit task
                         ▼
                 ┌──────────────────┐
                 │   pool queue     │
                 └──┬─────┬─────┬───┘
       postMessage  │     │     │
                    ▼     ▼     ▼
              Worker 1  Worker 2  Worker N    (own isolate, heap, loop)

   libuv threadpool (process-wide): fs, dns.lookup, crypto, zlib
   shared by the main thread and every worker

Because the libuv threadpool is shared by the whole process, a pool of workers that all call async crypto or zlib still competes for those same libuv threads.

┌────────────────────────────────────────────────┐
│ Main thread: own isolate, heap, event loop     │
│ HTTP handlers, I/O callbacks, task dispatch    │
└───────────────────────┬────────────────────────┘
                        │ enqueue task
                        ▼
    ┌───────────────────────────────────────┐
    │ Pool queue: pending tasks (FIFO)      │
    └──┬────────────────┬────────────────┬──┘
       │                │                │   postMessage()
       ▼                ▼                ▼   structured clone
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Worker 1     │ │ Worker 2     │ │ Worker N     │
│ own isolate  │ │ own isolate  │ │ own isolate  │
│ own heap     │ │ own heap     │ │ own heap     │
│ own loop     │ │ own loop     │ │ own loop     │
└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
       └────────────────┴────────────────┴──▶ results posted back to main

┌──────────────────────────────────────────────────────────────────────┐
│ libuv threadpool (process-wide): fs, dns.lookup, crypto, zlib        │
│ shared by the main thread and every worker                           │
└──────────────────────────────────────────────────────────────────────┘

The main thread queues tasks into a fixed pool of long-lived workers. Each worker has its own isolate, heap and event loop, and exchanges structured-clone messages with the main thread over postMessage, while one process-wide libuv threadpool serves the main thread and all workers.

How does it work? A request's path through a worker pool

Following one request from arrival to response makes the mechanics concrete:

  1. The request arrives on the main thread. The event loop runs your handler as usual. The handler reads and parses the body. This part still costs main-thread CPU, which is one reason you reserve headroom for it.
  2. The handler submits a task. It builds a plain-data payload (strings, numbers, plain objects, typed arrays) and hands it to the pool. The pool returns a promise, and the handler awaits it. While it waits, the main thread's event loop is free to run other callbacks.
  3. The pool dispatches or queues. If a worker is idle, the pool posts the payload to it. If every worker is busy, the task waits in the pool's queue until one frees up.
  4. The payload crosses the thread boundary. postMessage serializes the payload with structured clone on the main thread and deserializes it in the worker. Buffers in a transfer list are moved instead of copied. A SharedArrayBuffer is shared rather than copied.
  5. The worker runs the computation. The task function executes on the worker's own thread, with its own isolate and heap. The OS schedules this thread on a separate core when one is available, so the computation runs in parallel with the main thread and the other workers.
  6. The result travels back. The worker posts its return value, again via structured clone or transfer, to the main thread. A message event is queued on the main thread's event loop.
  7. The promise settles. The pool matches the result to the task that produced it and resolves that task's promise. If the task threw, or the worker died, the promise rejects instead.
  8. The handler responds. Execution resumes after the await, and the handler serializes and sends the response on the main thread.

Two consequences follow from this flow. First, the main thread only blocks during steps 1, 2, 4 (serialization on its side), 6 (deserialization on its side) and 8. That is why large payloads and very small tasks erode the benefit. Second, parallelism is capped by the number of cores the process can actually use. More workers than cores means the OS time-slices them, as covered in the pool-sizing section.

How do you implement a worker pool with Piscina in an HTTP handler?

With the raw API, the main thread calls new Worker('./hash.js') and listens for message, error and exit events. Inside the worker, the script reads input from workerData or a parentPort.on('message') handler and sends results back with parentPort.postMessage(). This is enough for one task. Under HTTP load it falls short. Each request needs its result matched to the right caller, so you end up adding task IDs. An uncaught exception emits error on the Worker object and terminates that thread, so you also need code to fail the pending request and start a new thread. Spawning a worker per request is the obvious shortcut, but each one starts its own V8 isolate and event loop. That startup cost can be larger than the work the worker does.

Piscina handles that bookkeeping for you. The task file exports a single function, sync or async, that receives the task payload and returns a value. In the main thread you create one Piscina instance at startup and point its filename option at that file. Each handler then calls await pool.run(payload). The returned promise resolves with the worker's return value.

Error handling follows from the promise. If the task function throws, pool.run() rejects with that error, and the handler's ordinary try/catch turns it into a 4xx or 5xx response. If a worker exits unexpectedly, its in-flight task rejects and the pool starts a replacement. Requests that come after the crash are unaffected.

The pool also queues tasks when every thread is busy, reuses threads across requests, and routes each result back to the promise that requested it. Without a pool, you would write and test all of that yourself.

// --- hash.js (task file: one exported function per task) ---
const { pbkdf2Sync } = require('node:crypto');

module.exports = ({ password, salt }) => {
  // A throw here rejects the caller's pool.run() promise
  if (typeof password !== 'string') throw new TypeError('password must be a string');
  // CPU-bound work runs on a pool thread, not the main event loop
  return pbkdf2Sync(password, salt, 310000, 32, 'sha256').toString('hex');
};

// --- server.js (main thread) ---
const http = require('node:http');
const path = require('node:path');
const Piscina = require('piscina');

// Create one pool at startup. Threads are reused, extra tasks are queued,
// and a worker that crashes is replaced.
const pool = new Piscina({ filename: path.resolve(__dirname, 'hash.js') });

http.createServer(async (req, res) => {
  let body = '';
  for await (const chunk of req) body += chunk;
  try {
    const { password } = JSON.parse(body);
    // Resolves with the task function's return value for this request only
    const hash = await pool.run({ password, salt: 'example-salt' });
    res.writeHead(200, { 'content-type': 'application/json' });
    res.end(JSON.stringify({ hash }));
  } catch (err) {
    // Bad JSON maps to 400. Errors thrown in the worker, or a worker that
    // exited mid-task, map to 500.
    console.error(err);
    res.writeHead(err instanceof SyntaxError ? 400 : 500);
    res.end(err instanceof SyntaxError ? 'invalid JSON' : 'internal error');
  }
}).listen(3000);

A Piscina task file and an HTTP handler that awaits pool.run(), where errors thrown in the worker or a crashed worker reject the promise and become an HTTP error response.

How many worker threads should a Node.js pool have?

Start from the CPUs the process can actually use, not the number of cores on the machine. os.availableParallelism(), where your Node.js version supports it, reports the CPUs the process can be scheduled on. It respects CPU affinity, but it does not read a container's CPU quota. A pod limited to two CPUs on a 32-core node can still see 32. On Linux, read the cgroup quota (for example cpu.max under cgroup v2) and use the smaller of the two numbers.

Then subtract headroom. The main thread still parses requests, serializes responses and runs the event loop. The libuv threadpool, four threads by default, handles file system access, DNS lookups, and crypto or zlib work. If you run cluster workers or several replicas on the same host, they all compete for the same cores. Divide the budget across them instead of giving each one a full-size pool.

Oversubscription does not add throughput for CPU-bound work. Once there are more runnable threads than cores, the kernel time-slices between them. That adds context switches and cache misses, and each task takes longer to finish. Median latency may barely change while p99 climbs.

Memory also grows with pool size. Each worker has its own V8 isolate and heap, so ten workers can need several times the memory of one process. Under a container memory limit, a large pool can trigger an OOM kill.

Treat the derived number as a starting point. Load-test at sizes around it, measure throughput and p99 latency, and choose the size where adding threads stops improving either one.

const os = require('node:os');
const fs = require('node:fs');

// cgroup v2 CPU quota, e.g. "200000 100000" = 2 CPUs, "max 100000" = unlimited.
// os.availableParallelism() does not see this, so read it directly.
function cgroupCpuLimit() {
  try {
    const [quota, period] = fs.readFileSync('/sys/fs/cgroup/cpu.max', 'utf8').trim().split(/\s+/);
    if (quota === 'max') return Infinity;
    return Number(quota) / Number(period);
  } catch {
    return Infinity; // not Linux, or cgroup v1: fall back to the scheduler's view
  }
}

function defaultPoolSize({ processesPerHost = 1, reserved = 1 } = {}) {
  // Respects CPU affinity; older Node versions only have os.cpus().
  const schedulable = typeof os.availableParallelism === 'function'
    ? os.availableParallelism()
    : os.cpus().length;

  // Use whichever limit is tighter: affinity or container quota.
  const usable = Math.min(schedulable, cgroupCpuLimit());

  // Cluster workers or replicas on the same host share these cores.
  const perProcess = usable / processesPerHost;

  // Reserve headroom for the main thread (event loop, parsing, serialization).
  // The libuv threadpool also competes; raise `reserved` if you do heavy fs/crypto/zlib work.
  return Math.max(1, Math.floor(perProcess - reserved));
}

// The derived value is only a starting point. Allow an override so load tests
// can sweep sizes around it and compare throughput and p99 latency.
const poolSize =
  Number(process.env.WORKER_POOL_SIZE) ||
  defaultPoolSize({ processesPerHost: Number(process.env.PROCESSES_PER_HOST) || 1 });

console.log(`worker pool size: ${poolSize}`);

Derives a starting worker pool size from the tighter of os.availableParallelism() and the cgroup v2 CPU quota, subtracts headroom for the main thread, splits the budget across processes on the same host, and allows an environment override for load-test tuning.

Data transfer: structured cloning, transferable ArrayBuffers and SharedArrayBuffer

The payload you send to a worker has its own cost, separate from the computation. By default, postMessage uses the structured clone algorithm. It walks the object graph, serializes it on the sending thread and deserializes it on the receiving one. For a multi-megabyte JSON body or image buffer, the main thread spends CPU on that serialization, which is the work you moved to a worker to avoid.

For binary data, put the ArrayBuffer in the transfer list. Piscina accepts a transfer list when you queue a task. The engine then moves ownership of the memory instead of copying it, so the transfer cost does not grow with the buffer's size. On the sending side the buffer becomes detached: its byteLength drops to zero and any typed-array views over it become empty. A common bug follows from this. The handler transfers the buffer and later reads it for logging, a checksum or a retry. That code then works on an empty buffer. It either throws or silently produces wrong results. Transfer the buffer only after the handler has finished using it. To keep the data, copy it before you send it.

SharedArrayBuffer lets both threads read and write the same memory. That suits large, long-lived data that many tasks read, or tight coordination between threads. In exchange, you handle synchronization yourself. You need Atomics for every concurrent read-modify-write, and Atomics.wait/notify for signalling. If you miss one of these, you get data races that only appear under load.

Mechanism Sender cost Sender's copy afterward Good fit
Structured clone Grows with payload size Intact Small, plain objects
Transfer list Does not grow with size Detached One-shot binary payloads
SharedArrayBuffer None per message Shared, needs Atomics Large shared state

For tiny tasks or very frequent tasks, the messaging round-trip can cost more than the work itself. Batch these tasks or run them inline.

How do you add backpressure, timeouts and memory limits to worker threads?

Set Piscina's maxQueue so the pool cannot buffer work without limit. When the queue is full, run() rejects immediately. Catch that rejection in the handler and return 503 with a Retry-After header, or 429 if the limit is per tenant. Failing fast is the goal. A request that waits 30 seconds behind thousands of others will usually time out at the client anyway, after it has already consumed memory.

Those responses only protect you if callers respect them. Clients and load balancers should treat 503 as "try elsewhere or later" and should not resubmit instantly. Retry policies need exponential backoff with jitter and a retry budget. Without them, every rejected request returns several times over, and a brief overload becomes a retry storm.

For timeouts, pass an AbortSignal (for example from AbortSignal.timeout()) to run(). Aborting a queued task simply removes it from the queue. Aborting a running task makes Piscina terminate that worker, which discards its warm state (JIT-optimized code, caches, loaded modules) and forces a respawn. If that cost matters, use cooperative cancellation instead: share a flag through a SharedArrayBuffer, and have the task code check it between chunks of work.

Set resourceLimits (maxOldGenerationSizeMb, maxYoungGenerationSizeMb) so a runaway task kills only its own worker, not the whole process.

On SIGTERM, stop accepting requests and wait for the pool's drain event, bounded by your grace period, before calling pool.destroy(). Anything still pending at that point is rejected.

Observability takes deliberate work. AsyncLocalStorage context, trace spans and logger bindings do not cross the thread boundary. Pass trace IDs in the task payload, create child spans inside the worker, and log the task ID alongside any error, because stack traces stop at the worker entry point.

How can worker threads save engineering time?

Worker threads do not make code simpler in general. They add a thread boundary, plain-data interfaces and harder debugging. They save engineering time when they replace a more expensive fix for the same problem. The savings come from a few places:

  • Fixing the problem where it is, instead of re-architecting. The usual alternative to a worker pool for a CPU-heavy endpoint is to split the work into a separate service or a queue-backed job system. That means a new deployable, a network protocol, retries, authentication between services, separate monitoring and a new failure mode to on-call for. A pool keeps the work in the same codebase, repository, deploy pipeline and language. The change is usually one task file plus a pool.run() call in the handler.
  • Reusing existing code and tests. If the expensive function is already a pure function of its inputs, it can move into a task file nearly unchanged. Its unit tests keep working because you can call the same function directly in tests, without starting a pool.
  • Letting a pool library carry the concurrency bookkeeping. Task-to-result matching, queuing, worker reuse, crash replacement, queue limits and cancellation are all things the raw worker_threads API leaves to you. Using a maintained pool such as Piscina means your team does not write, test and debug that machinery.
  • Fewer incidents caused by a blocked event loop. When one slow endpoint blocks the main thread, health checks fail, load balancers mark healthy instances as down, and unrelated endpoints time out. Those incidents are expensive to diagnose because the symptom (everything is slow) points away from the cause (one CPU-heavy handler). Moving the work off the main thread removes that class of cross-endpoint failure, and the event loop delay metric from the first section makes any regression visible.
  • Postponing horizontal scaling. If the process is mostly idle on I/O but blocked on one hot path, a pool lets a single process use the cores it already has. That can defer work on adding replicas, tuning autoscaling and handling the loss of in-process caches.

Whether the trade pays off depends on your situation, so estimate it before committing. Compare the change to the realistic alternative, not to doing nothing. If more replicas behind a load balancer would fix the problem with no code changes, that is usually cheaper still. If the job is heavy, bursty or needs another language, a separate service may cost less over time than stretching a worker pool to fit. The checklist in the next section helps make that call.

When should you avoid worker threads, and what are the alternatives?

Worker threads add little when the work already leaves the main thread. Async crypto.pbkdf2, crypto.scrypt, async zlib methods and most fs calls run on the libuv threadpool. Wrapping them in a worker adds a message hop and gains no parallelism. If those calls queue up, raise UV_THREADPOOL_SIZE (default 4) before you build a pool. Tasks that finish in well under a millisecond also lose: cloning the payload and posting the result can cost more than the computation. Batch them or run them inline.

Option Good fit Main cost
Inline Short, bounded work Blocks every request while it runs
libuv threadpool Built-in async crypto, zlib, fs Shared, small, easy to saturate
Worker pool Repeated CPU tasks in one service Plain-data interfaces, harder debugging
Cluster / replicas Uniform load, stateless handlers More memory, no shared in-process cache
Separate service Heavy or bursty jobs, other languages Network hop, deployment, failure modes

The latency gain has to justify the engineering cost. Worker tasks must take serializable inputs, so closures, class instances and database handles have to become plain data. Debugging also gets harder: errors cross threads and context does not propagate. Adding replicas behind a load balancer often removes the blocking problem with no code changes.

Decision checklist:

  • Does profiling show the main thread blocked by your own JavaScript, not by I/O?
  • Does each task run long enough to amortize transfer overhead?
  • Is the threadpool, not your code, the bottleneck? If so, tune it first.
  • Would more processes solve the problem with less change?
  • Is the job heavy enough that a separate queue-backed service is cleaner?

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to node.js worker threads, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.