Scaling Node.js: Cluster, Worker Threads and Horizontal Scale

Find the bottleneck before scaling Node.js

Scaling Node.js means adding execution capacity: more processes, more threads, or more machines. That helps only when your code is the constraint. If requests are slow because a database query takes 400 ms or a third-party API is rate-limiting you, ten more processes will open ten times the connections against the same slow dependency and often make things worse. So measure before you scale.

Slow Node.js services usually fall into one of four cases:

• Event-loop blocking: synchronous work such as large JSON.parse calls, regex backtracking or sync crypto holds the single JavaScript thread, so every other request waits.
• CPU-bound work: the process sits near 100% of one core because it is legitimately computing. Image resizing and report generation are typical examples.
• I/O wait: CPU is low and the event loop is responsive, but requests take a long time because they are awaiting disk or network.
• Downstream saturation: connection pools are exhausted, or a dependency's latency rises with load.

To tell these apart, start with event-loop delay. perf_hooks.monitorEventLoopDelay() reports a histogram of how late the loop runs scheduled work. A rising p99 there points to blocking or CPU saturation. Then capture a CPU profile with --cpu-prof or the inspector, and render it as a flame graph. The Node.js diagnostics guides cover both. Wide, flat frames in your own code show where the thread is spending its time.

Scale processes or threads when the evidence shows the event loop or a single process's CPU is the limit. If the time is going to a database or an external API, fix that first.

Copy// eld-monitor.js: log event-loop delay so you can tell blocking/CPU saturation
// apart from I/O wait or a slow downstream before deciding to scale.
const { monitorEventLoopDelay } = require('node:perf_hooks');

// Sample loop delay every 20 ms; values are recorded in nanoseconds.
const histogram = monitorEventLoopDelay({ resolution: 20 });
histogram.enable();

const toMs = (ns) => (ns / 1e6).toFixed(1);

const timer = setInterval(() => {
  // A rising p99 means synchronous work is holding the JS thread.
  // Slow requests with a flat p99 point to I/O wait or a saturated dependency.
  console.log(JSON.stringify({
    eld_mean_ms: toMs(histogram.mean),
    eld_p50_ms: toMs(histogram.percentile(50)),
    eld_p99_ms: toMs(histogram.percentile(99)),
    eld_max_ms: toMs(histogram.max),
  }));
  histogram.reset(); // Report each 10 s window on its own.
}, 10_000);

// Don't keep the process alive just for this monitor.
timer.unref();

// Load it before the app, then capture a CPU profile for a flame graph:
//   node --require ./eld-monitor.js --cpu-prof server.js
// --cpu-prof writes a .cpuprofile file when the process exits. Open it in
// Chrome DevTools to find the wide frames in your own code.

This preloaded script logs event-loop delay percentiles from perf_hooks.monitorEventLoopDelay() every 10 seconds and runs alongside --cpu-prof, so you can confirm the JavaScript thread is the bottleneck before you scale.

Who should use these scaling techniques

Each technique fits a different team and workload. The right one depends on where your service runs and on what your profile shows.

• Teams running Node.js on VMs or bare hosts without an orchestrator should consider cluster. It is the simplest way to use every core on one machine when request throughput is the limit and nothing else supervises the processes.
• Teams whose profiles show their own JavaScript holding the main thread should use worker threads. Examples include parsing large payloads, hashing, compression in pure JavaScript, and image or PDF generation. More processes would not help here, because each one would block the same way.
• Teams that need to run another program, such as a CLI tool, a binary from another language, or code they don't want sharing their heap, should use child processes.
• Teams on container platforms, or teams that need more capacity or availability than one machine gives, should scale horizontally. Run one process per container and let the orchestrator manage replicas.

Some teams should not reach for any of these yet. If your service is I/O-bound with low CPU and a flat event-loop delay, or if a database or external API is the constraint, adding processes or threads won't fix it. Work on queries, indexes, caching, connection pool sizing or the dependency first.

Why one Node.js process can't use all your cores

A Node.js process runs your JavaScript on one main thread. Request handlers, JSON parsing, template rendering and every callback the event loop dispatches all execute there, one at a time. While a handler runs, nothing else in that process's JavaScript can run.

Node.js does use other threads, through libuv. Network sockets are handled by the operating system's readiness mechanisms (epoll on Linux, kqueue on macOS, IOCP on Windows), so thousands of open connections cost little thread time. Work with no good non-blocking OS interface goes to libuv's thread pool. That pool has four threads by default and can be resized with the UV_THREADPOOL_SIZE environment variable. It handles most fs calls, dns.lookup, and the async forms of CPU-heavy built-ins such as crypto.pbkdf2, crypto.scrypt and zlib compression.

That offloading is narrow. It covers Node's own operations, not your code. A loop over a large array, a regex with heavy backtracking or a synchronous JSON.parse of a multi-megabyte body still runs on the main thread. On a 16-core machine, one busy process keeps roughly one core saturated with JavaScript while the other cores sit mostly idle. The runtime does not split your code across cores on its own.

Using more cores requires more than one JavaScript execution context, and the right kind depends on the bottleneck:

• Multiple processes on one host (cluster) suit request throughput limits.
• Worker threads suit individual CPU-heavy tasks.
• More hosts suit cases where one machine is no longer enough.

How Node.js scaling works

Every scaling option works the same basic way. It adds more JavaScript execution contexts, each with its own event loop and V8 heap. It also adds a way to route work to them. The options differ in what the context is and in what does the routing.

• Cluster: the context is a separate OS process running the same entry file. The primary process owns the listening socket and hands incoming connections to workers over IPC. Each worker serves its connections on its own event loop, so a host with N cores can run up to N event loops in parallel.
• Worker threads: the context is a thread inside the same process, with its own isolate and event loop. The main thread keeps handling I/O and sends discrete jobs to the workers with postMessage. Results come back the same way. Data is either copied (structured clone), transferred (ArrayBuffer in the transfer list), or shared (SharedArrayBuffer).
• Child processes: the context is any OS process, not necessarily Node.js. Communication goes over stdio or an IPC channel, and every message is serialized.
• Horizontal scale: the context is a whole instance on another container or machine. A load balancer distributes requests over the network and uses health checks to decide which instances receive traffic. Because any instance may serve any request, shared state has to live in external stores.

In all four cases you trade isolation for communication cost. The further apart two contexts are, the more expensive it is to pass data between them, and the less damage one failure can do to the others. The sections below cover how each mechanism behaves and what it costs.

How node cluster shares a port across processes

When a script calls cluster.fork(), Node starts a new process running the same entry file. Inside that file, cluster.isPrimary tells the code which role it has. The primary forks one worker per core, usually counted with os.availableParallelism(). Each worker then calls server.listen() as it normally would. The workers do not each bind the port. The primary owns the listening socket, and the workers ask it for access over the IPC channel. This is how several processes can share one port without "address in use" errors.

On every platform except Windows, the default policy is round-robin. The primary accepts each connection and passes the handle to the next worker. The alternative is to let workers accept connections directly, which leaves distribution to the OS scheduler. In practice that tends to send most connections to a few busy processes. You can choose the policy with cluster.schedulingPolicy or the NODE_CLUSTER_SCHED_POLICY environment variable. Round-robin distributes connections, not requests, so a client holding a long-lived keep-alive connection stays on one worker.

To recover from crashes, listen for the exit event on the cluster module and fork a replacement. Check worker.exitedAfterDisconnect first, so that a deliberate shutdown is not treated as a crash. Also add a short delay or a restart limit, so a worker that fails at boot doesn't loop forever.

Cluster fits VMs and bare hosts where nothing else supervises your processes. Avoid it in containers that an orchestrator already replicates, where it adds a second process manager under the first. Memory and CPU limits then apply to the whole group of workers rather than to one process. Health checks also become unreliable: they can pass while some workers are dead, because the primary is still alive. Run one process per container and let the orchestrator handle replicas and restarts.

Copyconst cluster = require('node:cluster');
const http = require('node:http');
const os = require('node:os');

const PORT = 3000;
const RESTART_DELAY_MS = 1000; // back off so a worker that crashes at boot doesn't spin the CPU
const MAX_RESTARTS = 10;       // stop replacing workers after this many unexpected exits
let restarts = 0;

if (cluster.isPrimary) {
  // Fork one worker per core this process is allowed to use.
  const count = os.availableParallelism();
  for (let i = 0; i < count; i++) cluster.fork();

  cluster.on('exit', (worker, code, signal) => {
    // Exits caused by worker.disconnect() or worker.kill() are deliberate, not crashes.
    if (worker.exitedAfterDisconnect) return;

    console.error(`worker ${worker.process.pid} died (code=${code}, signal=${signal})`);
    if (++restarts > MAX_RESTARTS) {
      console.error('restart limit reached, exiting primary');
      process.exit(1); // let an outer supervisor (systemd, etc.) decide what happens next
    }
    setTimeout(() => cluster.fork(), RESTART_DELAY_MS);
  });
} else {
  // Each worker calls listen() on the same port. The primary owns the listening
  // socket and gives connections to workers (round-robin by default except on Windows).
  http
    .createServer((req, res) => {
      res.end(`handled by pid ${process.pid}\n`);
    })
    .listen(PORT);
}

A primary process forks one worker per available core. All workers listen on port 3000. When a worker crashes, the primary replaces it after a delay and stops after a fixed restart limit, and it skips deliberate shutdowns by checking worker.exitedAfterDisconnect.

Offloading CPU-bound work with worker threads

Clustering multiplies event loops, but each one still blocks on CPU-heavy work. A 300 ms JSON parse or image resize stalls every request that process holds. Worker threads (node:worker_threads) move that work onto a separate thread with its own V8 isolate and event loop, so the main thread keeps serving I/O.

Good fits are parsing large documents, compression, hashing and image transformation. Some of this is already off the main thread: the async zlib and crypto functions, such as crypto.pbkdf2, run on the libuv threadpool. Use workers for pure-JavaScript CPU work that has no async variant.

A small pool

Create a fixed set of workers at startup, sized at or below the available cores. Give each worker a queue of jobs, and send work and results with postMessage. By default, messages are structured-cloned, which copies the data. To avoid the copy, pass an ArrayBuffer in the transfer list. Ownership moves to the worker and the sender's buffer becomes unusable. Use SharedArrayBuffer with Atomics when both sides need the same memory at once.

Costs

• Memory and startup. Each worker has its own heap and takes noticeable time to start. Reuse workers instead of spawning one per request. You can cap a worker's heap with the resourceLimits option.
• Cloning overhead. For small payloads, serializing and copying the data can cost more than the computation itself.
• No benefit for I/O-bound work. The main event loop already overlaps network and disk waits, so moving I/O into a worker adds overhead without making it faster.
• Shared failure. A thrown exception ends only the worker that threw it. A segfault in a native addon, however, kills the whole process, including every thread in it.

Copy// pool.js: one file runs as both the main thread and the workers
const { Worker, isMainThread, parentPort } = require('node:worker_threads');
const { availableParallelism } = require('node:os');
const { createHash } = require('node:crypto');

if (!isMainThread) {
  // Worker: synchronous CPU work that blocks only this thread
  parentPort.on('message', ({ id, buf }) => {
    const digest = createHash('sha256').update(new Uint8Array(buf)).digest('hex');
    parentPort.postMessage({ id, digest });
  });
} else {
  const idle = [], queue = [], pending = new Map();
  let nextId = 0;
  // Fixed pool, created once at startup. Leave one core for the main event loop.
  const workers = Array.from({ length: Math.max(1, availableParallelism() - 1) }, () => {
    const w = new Worker(__filename, { resourceLimits: { maxOldGenerationSizeMb: 256 } });
    w.on('message', ({ id, digest }) => {
      pending.get(id).resolve(digest); pending.delete(id);
      idle.push(w); drain();
    });
    // A JS exception kills only this worker. Fail its job. A real pool would respawn it.
    w.on('error', (err) => { pending.get(w.jobId)?.reject(err); pending.delete(w.jobId); });
    idle.push(w);
    return w;
  });
  function drain() {
    while (idle.length && queue.length) {
      const w = idle.pop(), job = queue.shift();
      w.jobId = job.id;
      // The transfer list moves the ArrayBuffer instead of structured-cloning it
      w.postMessage({ id: job.id, buf: job.buf }, [job.buf]);
    }
  }
  const run = (buf) => new Promise((resolve, reject) => {
    const id = nextId++; pending.set(id, { resolve, reject });
    queue.push({ id, buf }); drain();
  });

  const payload = new Uint8Array(64 * 1024 * 1024).fill(7).buffer; // 64 MiB placeholder
  run(payload).then((digest) => {
    console.log(digest, 'sender byteLength now:', payload.byteLength); // 0: buffer was transferred
    workers.forEach((w) => w.terminate());
  });
}

A fixed-size worker pool that queues hashing jobs, sends each buffer through postMessage's transfer list to avoid a copy, and caps each worker's heap with resourceLimits.

Cluster vs worker threads vs child processes vs horizontal scale

Each option draws its isolation boundary in a different place. Where that boundary sits determines how much data costs to move across it and how much breaks when something on the other side fails.

Choose based on the bottleneck you measured. If profiling shows the event loop blocked by your own computation, use worker threads. More processes would each block the same way. If one process is I/O-bound and leaves cores idle, use cluster or run one process per core under your orchestrator. If the work is a separate program, spawn it as a child process. If one host can't handle peak load or survive its own failure, scale horizontally.

These options combine. A common shape is several stateless instances behind a load balancer, each with a small worker pool for CPU-heavy requests. The Node.js docs compare these concurrency models in their Concurrency section.

Horizontal scaling behind a load balancer

Run one Node.js process per container or VM and let the orchestrator handle replica count, restarts and placement. In-process clustering on top of that would hide crashes and memory use from the scheduler. An external load balancer spreads requests across instances. Sessions, caches and queues live in shared stores such as a database or cache cluster, so any instance can serve any request.

Health checks and readiness

Expose two separate signals. Liveness answers "is the process stuck?" Readiness answers "should I get traffic right now?" Report not-ready during startup until connection pools are warm, and again as soon as shutdown begins. A failing dependency should affect readiness, not liveness, so the orchestrator doesn't restart every instance during a database blip.

Graceful shutdown

On SIGTERM, take these steps in order:

1. Flip readiness to failing and wait briefly so the load balancer deregisters the instance.
2. Call server.close() to stop accepting new connections while in-flight requests finish.
3. Close database pools.
4. Exit before the orchestrator's kill deadline.

Start node directly or through a minimal init. Wrapper scripts may not forward the signal.

Keep-alive alignment

Set Node's keepAliveTimeout longer than the load balancer's idle timeout. Otherwise Node closes a socket the balancer is about to reuse, which produces intermittent 502s. Keep headersTimeout above keepAliveTimeout.

Sticky sessions

WebSockets pin a client to one instance for the life of the connection. This skews load: new instances receive only new connections. Every deploy also severs the existing connections, so clients need jittered reconnects to avoid a reconnect storm.

Copyconst http = require('node:http');
const db = require('./db'); // placeholder: a pool exposing connect() and end()

let ready = false; // flips true once pools are warm, false again on SIGTERM

const server = http.createServer((req, res) => {
  if (req.url === '/livez') return res.end('ok'); // process is responsive
  if (req.url === '/readyz') {
    res.statusCode = ready ? 200 : 503; // controls whether the LB routes traffic here
    return res.end();
  }
  // While draining, ask clients to drop their keep-alive sockets after this response
  if (!ready) res.setHeader('Connection', 'close');
  res.end('hello');
});

// Assumes a load balancer idle timeout of 60s; Node must outlive it
server.keepAliveTimeout = 65_000;
server.headersTimeout = 66_000; // must stay above keepAliveTimeout

server.listen(3000, async () => {
  await db.connect(); // report ready only after the pool is warm
  ready = true;
});

process.once('SIGTERM', async () => {
  ready = false; // 1. fail readiness
  // Wait long enough for the LB to observe failing probes and deregister
  await new Promise((resolve) => setTimeout(resolve, 10_000));
  // Safety net: exit before an assumed 30s orchestrator grace period
  setTimeout(() => process.exit(1), 15_000).unref();
  // 2. stop accepting connections; callback fires when in-flight requests finish
  server.close(async () => {
    await db.end(); // 3. close database pools
    process.exit(0); // 4. clean exit
  });
  server.closeIdleConnections(); // idle keep-alive sockets would otherwise hold close() open
});

A Node.js HTTP server with separate liveness and readiness endpoints, keep-alive timeouts set above an assumed 60-second load balancer idle timeout, and a SIGTERM handler that fails readiness, waits for deregistration, drains connections, closes the database pool and exits before the kill deadline.

Stateless processes, memory sizing and cost

Any state held in process memory becomes wrong once a second copy runs. Per-process caches diverge, so two requests for the same key can return different data. Rate-limit counters undercount: a limit of 100 per minute enforced in each of eight processes allows 800. Scheduled jobs fire once per instance, sending duplicate emails or double-charging. Move sessions, caches and counters into a shared store, and run scheduled work as a single job or behind a distributed lock.

Memory has to be budgeted per process, not per container. Each cluster worker and each worker thread has its own V8 heap. Cap the heap with --max-old-space-size, or with resourceLimits for worker threads, and leave headroom for buffers and native memory outside the heap.

Illustrative estimate (assumed numbers, not a benchmark):

• Container memory ≥ processes × (heap cap + off-heap headroom). With 4 processes × (384 MiB + 128 MiB), you need 2,048 MiB.
• Database connections = instances × processes per instance × pool size. With 10 × 4 × 10, you open 400 connections. If the database is assumed to allow 200, scale-out fails at the database rather than at the app tier. Shrink the per-process pools or put a connection pooler in front of the database.

These choices also drive cost. Profiling first keeps you from building infrastructure you don't need. An orchestrator restarts and replaces processes, so you don't have to write and maintain your own supervision code. Stateless processes make deploys and scale events routine, so they stop costing engineering time.

How the right scaling approach saves engineering time

Most of the time lost on Node.js scaling goes to building the wrong thing, or to debugging problems the architecture created. The practices in this article cut both.

• Measuring first avoids wasted projects. Event-loop delay and a CPU profile take minutes to collect. They tell you whether to scale at all, and which mechanism to use, before anyone builds a cluster setup or a worker pool. Without them, a team can spend a sprint adding processes to a service whose real problem is a slow query, and then have to debug the connection exhaustion that follows.
• Matching the mechanism to the bottleneck avoids rework. Moving one CPU-heavy function into a worker pool is a local change, and the rest of the service stays the same. Clustering a service that blocks on CPU does not fix the latency spikes. It only spreads them across more processes, and someone still has to diagnose them later.
• Letting the platform supervise processes removes code you would otherwise own. In containers, one process per instance means restarts, replica counts, placement and rolling deploys come from the orchestrator. You don't write or maintain restart loops, back-off logic or a primary process. Crashes and memory use are also visible to the scheduler and your monitoring, instead of hidden behind a primary that still looks healthy.
• Correct health checks and graceful shutdown reduce incident time. Separate liveness and readiness probes stop the orchestrator from restarting every instance during a dependency outage. Draining on SIGTERM and aligning keep-alive timeouts remove a common source of intermittent 502s during deploys. These errors are hard to reproduce and slow to trace, so preventing them saves the most time.
• Stateless processes make scaling a configuration change. When sessions, caches, counters and scheduled jobs live in shared stores, adding instances means changing a replica count. You avoid the duplicate-job and divergent-cache bugs that show up only after scale-out, which are often found in production.
• Doing the capacity arithmetic early prevents surprise limits. A short calculation of heap caps per process and of total database connections shows memory and connection limits before an incident does, as in the illustrative estimate above.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to scaling nodejs, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.