Why do CPU-bound tasks block the event loop for every request?
A CPU-bound task is one where the time goes to computation, not to waiting on a socket or disk. In API code this usually means hashing passwords with a slow algorithm, parsing or validating large JSON payloads, rendering templates, resizing images, compressing responses, or running scoring logic over a large in-memory dataset.
Node.js runs your JavaScript on a single main thread, and the event loop only advances when that thread is free. I/O is cheap because Node hands it to the operating system or the libuv thread pool and moves on. A synchronous computation keeps the thread until it finishes. While it runs, no callbacks fire, no timers run, and no new connections are read. Every in-flight request waits behind it, so one slow endpoint raises latency for all endpoints.
Here is an illustrative estimate with assumed inputs. A request spends 80 ms of synchronous CPU, and the service handles 10 of them per second alongside fast health checks. A health check that arrives during one of those windows waits up to 80 ms before Node even reads it. The median looks fine, but p99 degrades across the whole service.
Measure before you change the architecture. monitorEventLoopDelay from perf_hooks samples how late the loop runs. It returns a histogram that reports percentiles in nanoseconds. Export its p99 and max next to your request latency percentiles. If spikes in loop delay match spikes in tail latency, the cause is CPU on the main thread. If they don't match, look at downstream dependencies first.
This article is for backend and platform engineers whose request paths do significant work in JavaScript.
// Reproduce the problem: one CPU-heavy route plus a cheap health check,
// with event loop delay sampled so it can be compared to tail latency.
const http = require('node:http');
const crypto = require('node:crypto');
const { monitorEventLoopDelay } = require('node:perf_hooks');
// Sample loop lateness every 20 ms. Histogram values are in nanoseconds.
const loopDelay = monitorEventLoopDelay({ resolution: 20 });
loopDelay.enable();
const server = http.createServer((req, res) => {
if (req.url === '/slow') {
// Synchronous key derivation holds the main thread until it finishes.
// Any /health request that arrives meanwhile is not even read yet.
const key = crypto.pbkdf2Sync('placeholder-password', 'placeholder-salt', 600000, 32, 'sha256');
res.end(key.toString('hex'));
return;
}
if (req.url === '/health') {
res.end('ok');
return;
}
res.statusCode = 404;
res.end();
});
// Report per-interval loop delay in milliseconds. In production, export
// these as metrics next to request latency p99 instead of logging.
const toMs = (ns) => (ns / 1e6).toFixed(1);
setInterval(() => {
console.log(
`loop delay p50=${toMs(loopDelay.percentile(50))}ms ` +
`p99=${toMs(loopDelay.percentile(99))}ms max=${toMs(loopDelay.max)}ms`
);
loopDelay.reset(); // start a fresh window so spikes line up with latency graphs
}, 5000).unref();
server.listen(3000);
A minimal HTTP server where a synchronous pbkdf2Sync call on one route stalls every request, with monitorEventLoopDelay reporting per-interval p50, p99 and max loop delay so you can correlate it with tail latency.
Who should use worker threads or child processes?
Offloading is for teams running Node.js APIs where measurement shows that synchronous JavaScript on the main thread is driving tail latency. Typical signs are event loop delay spikes that line up with p99 spikes, a mix of fast and slow endpoints in one service, and health checks that time out under load even though the process is not waiting on any dependency.
The approaches fit different needs:
- Worker threads suit engineers who need CPU work to finish inside the request but who still want one deployable process. Examples are validation, parsing, templating or scoring written in JavaScript and running for milliseconds to a few seconds.
- Child processes suit work that runs an external binary, depends on native code you don't trust not to crash, or needs OS-enforced memory limits per task.
- A separate service or job queue suits platform teams whose CPU demand grows independently of request traffic, or whose tasks run longer than a request timeout.
Some services don't need either. If the service is mostly I/O-bound, the event loop is already doing its job, and threads add overhead without benefit. Check whether an asynchronous built-in already exists before building a pool. For example, crypto.pbkdf2 (as opposed to pbkdf2Sync) runs on the libuv thread pool and does not block the main thread. If the hot path is a library you can swap for a non-blocking variant, that is usually the smaller change.
How does it work? Worker threads and child processes in Node.js
A worker thread runs a separate V8 isolate with its own heap and event loop, but it lives inside the same OS process as the main thread. Workers cannot touch each other's JavaScript objects. They communicate with postMessage, which copies data using the structured clone algorithm. Copying a large payload costs CPU on both sides, so binary data should go in the transfer list. A transferred ArrayBuffer moves to the receiver without a copy, and the sender's reference becomes detached and unusable. A SharedArrayBuffer goes further: both threads see the same memory. Plain reads and writes to it can race, so any coordination has to use Atomics. A forgotten Atomics call produces bugs that pass tests and appear only under load.
A child process is a separate OS process with its own heap and runtime:
spawnstreams stdio.execruns a command through a shell and buffers its output.forkstarts a new Node.js process with an IPC channel attached.
Messages over that channel are serialized, written to the channel, and parsed on the other side. Every fork boots a full runtime, so startup takes longer and each process holds more resident memory than a worker. In exchange, the OS isolates the processes: a crash or memory leak in one does not take down the parent.
In both models, the request flow is the same. The main thread receives the request, packages the input as a message, and hands it to another thread or process. It goes back to serving other connections while the computation runs elsewhere. When the result message arrives, a callback or promise resolves on the main thread's event loop, and the handler writes the response. The main thread stays free because it only does the cheap parts: parsing the request, sending the message, and writing the reply.
Worker threads (one process) Child processes (many processes)
+--------------------------------+ +-------------+ +-------------+
| Main isolate Worker isolate| | Parent node | | Child node |
| [heap][loop] <-> [heap][loop] | | [heap][loop]| <==> | [heap][loop]|
| postMessage (clone/transfer) | +-------------+ IPC +-------------+
| SharedArrayBuffer + Atomics | separate address spaces,
+--------------------------------+ serialized messages
WORKER THREADS: one OS process, multiple V8 isolates
+--------------------------------------------------------------+
| OS process (one PID) |
| +--------------------+ +--------------------+ |
| | Main thread | postMessage | Worker thread | |
| | V8 isolate | ------------> | V8 isolate | |
| | [heap] [loop] | <------------ | [heap] [loop] | |
| +--------------------+ clone or +--------------------+ |
| | transfer | |
| | +-------------------------+ | |
| +----> | SharedArrayBuffer | <-+ |
| | coordinate via Atomics | |
| +-------------------------+ |
+--------------------------------------------------------------+
CHILD PROCESSES: separate OS processes and address spaces
+------------------------+ +------------------------+
| Parent process (PID A) | IPC channel | Child process (PID B) |
| Node.js runtime | ==============> | Node.js runtime |
| [heap] [loop] | <============== | [heap] [loop] |
+------------------------+ serialize -> +------------------------+
write -> parse
spawn: streams stdio exec: shell, buffers output
fork: new Node.js process with IPC channel attached
Worker threads share one OS process and exchange data by cloning, transferring, or sharing memory guarded by Atomics, while child processes each run a full runtime in a separate address space and talk through serialized IPC messages.
Node.js worker threads vs child process: side-by-side comparison
| Dimension | Worker threads | Child processes |
|---|---|---|
| Startup cost | New V8 isolate and Node environment, but no new OS process | Full OS process plus a fresh Node runtime boot |
| Memory overhead | Separate heap per worker inside one process | Separate heap plus duplicated runtime and process overhead |
| Communication | postMessage with structured clone; ArrayBuffers can be transferred without copying |
IPC channel; JSON by default, or structured clone with serialization: 'advanced' in fork |
| Crash containment | Native segfault or abort() kills the whole process |
Failure stays in the child; parent sees an exit event |
| Per-task memory limit | resourceLimits per worker; a JS heap OOM ends only that worker |
--max-old-space-size per child, enforced by an isolated process |
| Non-JavaScript executables | No | Yes, via spawn / execFile |
| Shared memory | SharedArrayBuffer with Atomics |
None; all data is copied |
| Debugging and observability | One PID; per-thread CPU needs thread-level tooling | Each child has its own PID, inspector port and logs |
Worker threads isolate the JavaScript heap. They do not isolate the process. A segfault in a native addon terminates every worker and the main thread together, because they share one address space. Process-wide limits behave the same way. A container memory cgroup, the file descriptor ulimit and the kernel OOM killer all see one process, so a single runaway worker can take down the whole API. When a child process fails, only that child dies, and the parent can respawn it.
The trade-off runs the other way for data exchange. Workers can share or transfer buffers, so moving large inputs stays cheap. With child processes, every payload is serialized, crosses the IPC channel and is deserialized again, and that per-message cost dominates when tasks are small and frequent.
Why should a worker pool be the default, and how do you implement one with piscina?
Why pooling beats spawning per request
Creating a worker for every request puts thread startup on the request path. Each worker boots a fresh V8 isolate, loads its modules and compiles its code before it does any useful work, so the request pays that cost every time. Per-request spawning also has no upper bound: a traffic spike creates as many threads as there are in-flight requests, they compete for the same cores, and latency rises for all of them. A pool starts a fixed number of workers once and reuses them, so concurrency is capped by design.
The piscina model
Piscina manages a fixed set of workers that pull from a shared task queue. Callers submit a task and get back a promise. Each submission can carry an AbortSignal, so a timed-out request also cancels its queued or running work. The queue can be bounded, and when it is full the pool rejects new tasks immediately. That is backpressure: you shed load at the edge instead of building a backlog nobody will wait for.
The worker file is a module that exports one function, takes the task payload and returns a result. The API handler checks queue depth first and returns 503 with Retry-After if the pool is saturated. Otherwise it submits the task with a timeout signal and maps an abort to 504.
What doesn't cross the thread boundary
Errors thrown in a worker are cloned to the main thread. Custom error classes and non-standard properties may not survive the copy, so return structured error codes instead of relying on instanceof. AsyncLocalStorage context stays on the main thread. Pass the request ID and serialized trace context in the payload, and rebuild the logger and span inside the worker.
// worker.mjs: runs inside a pooled thread, one exported function per task
export default function ({ requestId, traceparent, n }) {
// AsyncLocalStorage does not cross threads; rebuild log context from the payload
const log = (msg) => console.log(JSON.stringify({ requestId, traceparent, msg }));
if (!Number.isInteger(n) || n < 0) return { ok: false, code: 'BAD_INPUT' };
let acc = 0;
for (let i = 0; i < n; i++) acc = (acc + i * i) % 1_000_007; // CPU-bound placeholder
log('task done');
return { ok: true, result: acc }; // structured result instead of custom error classes
}
// server.mjs: one pool for the whole process, created at startup
import http from 'node:http';
import Piscina from 'piscina';
const MAX_QUEUE = 64;
const pool = new Piscina({
filename: new URL('./worker.mjs', import.meta.url).href,
maxThreads: 4, // fixed concurrency, roughly matched to available cores
maxQueue: MAX_QUEUE, // bounded queue
});
http.createServer(async (req, res) => {
// Backpressure: shed load before queuing work nobody will wait for
if (pool.queueSize >= MAX_QUEUE) {
res.writeHead(503, { 'Retry-After': '1' }).end();
return;
}
const n = Number(new URL(req.url, '').searchParams.get('n'));
const signal = AbortSignal.timeout(2000); // cancels queued or running work
try {
const out = await pool.run(
{ requestId: req.headers['x-request-id'], traceparent: req.headers['traceparent'], n },
{ signal });
res.writeHead(out.ok ? 200 : 400, { 'content-type': 'application/json' }).end(JSON.stringify(out));
} catch (err) {
res.writeHead(signal.aborted ? 504 : 500).end();
}
}).listen(3000);
This example shows a minimal piscina worker and HTTP handler. The handler caps concurrency with a fixed pool, returns 503 with Retry-After when the queue is full, and cancels work through an AbortSignal timeout that maps to 504. Request and trace IDs travel to the worker inside the task payload.
How many workers should a pool have?
Size the pool from the CPU you can actually use, not from request volume. Each worker runs CPU-bound code, so a worker only makes progress while it holds a core. If you add workers beyond the available cores, the operating system time-slices them. Every task takes longer in wall-clock terms, and total throughput stays the same. You have stopped blocking the event loop, but now requests wait in a run queue that you cannot observe.
Containers make core counting unreliable. A pod with a 2-CPU quota on a 32-core node may still report 32 cores to os.cpus(). If you size the pool to 32, the threads use up the quota early in each scheduling period. The kernel then throttles all of them, including the main thread, until the next period starts. Mean latency can look fine while p99 latency jumps by the length of the throttled window. Read the cgroup quota, or set the pool size explicitly through configuration. Leave headroom for the main thread.
| Pool size relative to usable CPU | Effect |
|---|---|
| Below | Idle CPU; queue grows earlier than necessary |
| Equal (minus main thread) | Maximum throughput, predictable latency |
| Above | Time-slicing or quota throttling; worse tail latency |
Illustrative estimate (assumed inputs, not measurements)
- Assumptions: a 4-CPU quota, 20 ms of CPU per task, and one core reserved for the main thread.
- Pool size: 3 workers.
- Ceiling: 3 × (1000 / 20) = 150 tasks/s.
- Target of 120 tasks/s: about 80% utilization, which is stable.
- Above 150 tasks/s, the queue grows without limit. At 180 tasks/s it gains 30 tasks per second. After 10 s, 300 tasks are queued, so a new task waits about 2 s before it starts.
When should you avoid worker threads?
Serialization is the hidden tax
Every postMessage call runs the structured clone algorithm. The message is copied out of the main thread's heap and rebuilt in the worker's heap, and the result comes back the same way. For a task that computes for a fraction of a millisecond, that round trip can cost more than the work itself. The same is true for a cheap transform over a large object graph. Before offloading anything, compare payload size against compute time. If cloning the input and output takes about as long as the computation, the worker adds latency and frees almost no main-thread time.
Transferring an ArrayBuffer avoids the copy, but ownership moves with it. After the transfer, the buffer is detached on the sending side and its length drops to zero. Code that still reads it after posting will fail or see no data.
When child processes earn their cost
A child process is a full Node runtime with its own heap and startup time. It is worth that cost in three cases:
- Untrusted or crash-prone native code. A segfault in a native addon running in a worker kills the entire process, including every in-flight request. In a child, it kills only that child.
- External binaries. If the work is done by a non-JavaScript program, spawn that program directly.
- Hard per-task memory caps. A worker's
resourceLimitsoption limits only the V8 heap. OS-level limits apply to whole processes.
Budget for this carefully. In a memory-constrained container, every child counts against the same limit, so the OOM killer can terminate the container. Each new replica also pays the children's startup time before it can serve traffic, which slows scale-up.
// clone-tax.js: run with `node clone-tax.js`
const { Worker, isMainThread, parentPort } = require('node:worker_threads');
if (isMainThread) {
// structuredClone uses the same algorithm as postMessage, so it gives a
// rough measure of the copy cost without starting a worker.
const payload = {
rows: Array.from({ length: 200_000 }, (_, i) => ({ id: i, score: i % 97 })),
};
let t = performance.now();
structuredClone(payload); // one direction only; a worker round trip also clones the result
const cloneMs = performance.now() - t;
t = performance.now();
payload.rows.reduce((sum, r) => sum + r.score, 0); // the actual work
const computeMs = performance.now() - t;
console.log(`clone ${cloneMs.toFixed(1)} ms vs compute ${computeMs.toFixed(1)} ms`);
// If cloning costs about as much as computing, keep the work on the main thread.
// Transferring avoids the copy, but the sender loses the buffer.
const worker = new Worker(__filename);
const buf = new ArrayBuffer(16 * 1024 * 1024);
worker.postMessage(buf, [buf]); // second argument is the transfer list
console.log(`sender byteLength after transfer: ${buf.byteLength}`); // 0: detached
worker.once('message', (len) => {
console.log(`worker received ${len} bytes`);
worker.terminate();
});
} else {
// The worker now owns the memory.
parentPort.once('message', (buf) => parentPort.postMessage(buf.byteLength));
}
Measures structured clone cost against compute time for a cheap task over a large object graph, then shows that a transferred ArrayBuffer is detached on the sending side.
What are the alternatives, and how do you choose?
Those startup and memory costs are part of a broader trade-off. Each option moves the CPU work further from the request, which buys isolation and independent scaling but adds engineering and operational cost.
| Option | Fits | Main cost |
|---|---|---|
| Worker thread pool | Short tasks (milliseconds to a few seconds) that must finish inside the request | Shared process: a native crash or memory leak takes down the API |
| Child process pool | Untrusted or crash-prone code, separate binaries, hard memory isolation | Slower startup, serialization over IPC, per-child memory |
| Dedicated service or job queue | Long-running or sustained CPU load, or work that outlasts request timeouts | Deployments, queue infrastructure, retries, idempotency, result delivery |
Build a single shared pool module rather than spawning threads or processes per request. Without one, each endpoint ends up with its own timeout, error, and concurrency handling, often slightly wrong. A shared module gets those decisions right once, and every caller inherits the fixes.
Budget for harder debugging. Stack traces stop at the thread or process boundary. The profiler and inspector attach to each isolate or process separately, so you have to correlate logs by task ID yourself.
A separate service is worth its overhead when CPU demand scales differently from request traffic. Examples are nightly batch spikes, or a few tenants producing most of the load. In those cases, scaling API replicas to absorb the CPU work wastes capacity.
How can this save engineering time?
The main time savings come from making latency problems easy to diagnose and from making the fix a reusable component instead of per-endpoint work.
- Faster diagnosis of tail-latency incidents. With event loop delay exported next to request latency, an on-call engineer can see within one graph whether a p99 spike comes from main-thread CPU or from a downstream dependency. Without that signal, teams often spend the incident checking databases and networks that turn out to be healthy.
- One pool module instead of per-endpoint fixes. Timeouts, cancellation, backpressure, error mapping and trace propagation are written and reviewed once. Moving another slow handler off the main thread then means writing a worker function and calling
pool.run, not redesigning concurrency for that route. - Avoiding a new service when an in-process pool is enough. A worker pool ships with the existing deployment. It needs no new pipeline, queue infrastructure, retry semantics or result-delivery mechanism. For short tasks that must finish inside the request, that removes a whole project from the roadmap. The choice table above shows when the extra service actually pays for itself.
- Capacity planning as arithmetic. A fixed pool sized from the CPU quota gives a throughput ceiling you can calculate, as in the illustrative estimate above. Teams can predict when to add replicas instead of discovering the limit during an outage.
- Fewer bugs that appear only under load. A bounded queue that sheds load with 503 and cancels expired work with an
AbortSignalprevents the slow pile-ups that are hard to reproduce in tests. Structured error codes and payload-carried request IDs make failures inside workers traceable without attaching a debugger to each thread. - Not building what you don't need. Measuring clone cost against compute time first, and checking for async built-ins such as
crypto.pbkdf2, keeps teams from building pool infrastructure for work that never needed to leave the main thread.
Checklist before choosing
- How long does one task run at p99, compared with your request timeout?
- Must the caller wait for the result, or can it poll or receive a callback?
- Can a bug in the task crash or corrupt the API process?
- Does CPU load track request volume, or does it vary independently?
- Who will own the extra queue or service in production?
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to node.js worker threads vs child process, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.