Node.js Memory Leak Debugging: Heap Snapshots to Root Cause

Is it a leak or normal V8 behavior?

Memory that rises after a deploy is not, on its own, evidence of a leak. V8 allocates into the young generation, promotes surviving objects to the old generation, and collects lazily. It grows the heap to suit the workload rather than returning memory as soon as it can. A healthy API server therefore shows a sawtooth, where usage climbs between collections and drops after each one. It may also show a plateau, where memory rises during warm-up while caches, compiled code and connection pools fill, then levels off.

A leak shows up in the post-GC floor. Under steady traffic, the lowest point after each major collection keeps rising over many GC cycles. That happens because something still holds references to objects that should have died, so the collector cannot free them.

To see the floor, sample process.memoryUsage() on a fixed interval and log heapUsed and rss together. A ten-second sample interval is enough to see the trend over an hour of load. Compare the two series:

The rest of this guide covers the third row. If you are in the fourth, a heap snapshot will look clean and send you in the wrong direction.

Copy// memory-sampler.js: load early in the API server, e.g. require('./memory-sampler')
const SAMPLE_INTERVAL_MS = 10_000; // 10s is enough to see a trend over an hour of load
const toMB = (bytes) => Math.round((bytes / 1024 / 1024) * 10) / 10;

let baseline = null;

const timer = setInterval(() => {
  const { heapUsed, heapTotal, rss, external, arrayBuffers } = process.memoryUsage();
  if (!baseline) baseline = { heapUsed, rss };

  // One JSON line per sample so you can plot heapUsed and rss on the same chart.
  // external and arrayBuffers track Buffer and other off-heap memory that
  // heap snapshots do not show.
  console.log(JSON.stringify({
    ts: new Date().toISOString(),
    heapUsedMB: toMB(heapUsed),
    heapTotalMB: toMB(heapTotal),
    rssMB: toMB(rss),
    externalMB: toMB(external),
    arrayBuffersMB: toMB(arrayBuffers),
    // Growth since the first sample. Judge the trend of the lows (post-GC floor),
    // not single peaks, because a sawtooth is normal.
    heapDeltaMB: toMB(heapUsed - baseline.heapUsed),
    rssDeltaMB: toMB(rss - baseline.rss),
  }));
}, SAMPLE_INTERVAL_MS);

// Do not keep the process alive just for sampling.
timer.unref();

Logs heapUsed, rss and off-heap counters from process.memoryUsage() every 10 seconds as JSON lines so you can tell a rising JS heap floor apart from RSS growing on its own.

Who should use heap snapshot debugging

This workflow is for engineers who own long-running Node.js processes: API servers, queue workers, WebSocket gateways, and other services that stay up for hours or days and handle many requests in one process. In those processes, a few retained objects per request add up until the container hits its memory limit and gets killed or restarted.

It is most useful to:

• Backend and service owners who see a rising post-GC floor on their dashboards and need to find which code path is responsible.
• On-call and SRE engineers who need evidence they can hand to the owning team, such as a constructor name and a retainer chain, rather than a graph that only shows memory going up.
• Library and platform maintainers whose code runs inside other teams' services. A leak in shared middleware, instrumentation, or a client library shows up in every consumer, and a snapshot diff can show the shared module as the retainer.

You need to be able to run the service with production-like configuration somewhere you control, start it with extra Node flags, and open .heapsnapshot files in Chrome DevTools. You do not need deep knowledge of V8 internals. Reading the Summary, Comparison, and Retainers views is enough for most leaks.

Short-lived processes such as CLI tools, build scripts, and serverless handlers that exit after each invocation rarely benefit. The process ends before retained memory matters.

When to use heap snapshot debugging

Use heap snapshots when the sampler above puts you in the "JS heap leak" row: under steady load, the post-GC heapUsed floor keeps rising and rss rises with it. Typical triggers:

• Containers are OOM-killed or restarted on a regular schedule, and the time to failure gets shorter as traffic grows.
• Memory graphs show a staircase after a specific deploy, which points to a recent code or dependency change.
• Latency gets worse as the process ages, because a fuller old generation makes major collections run longer and more often.
• Node prints MaxListenersExceededWarning, which often means listeners are being added per request and never removed.

Do not reach for snapshots in these cases:

• Sawtooth or warm-up plateau. There is nothing to find.
• RSS rising while heapUsed is flat. The growth is off-heap, and snapshots cover only the JS heap. Start with external and arrayBuffers, then native addons and the allocator.
• You cannot reproduce growth yet. Without a repeatable load, diffs mix the leak with noise. Build the reproduction first, as described below.
• High memory that is stable. A large but flat heap is a sizing or efficiency question. A sampling heap profiler, which shows where allocations happen, is a better fit than a retention diff.

How heap snapshot debugging works

A heap snapshot is a complete graph of the JavaScript heap at one moment. Each node is an object (a plain object, array, string, closure, compiled code, and so on) with its constructor name and shallow size. Each edge is a reference: a property name, an array index, a closure's captured variable, or an internal V8 link. The graph also records GC roots, the starting points the collector uses to decide what is alive, such as the global object, active stack frames, and handles held by Node internals.

DevTools uses that graph to compute two things you rely on:

1. Reachability and retained size. An object is alive if a path of edges leads to it from a root. Its retained size is the memory that would be freed if it were removed, which is how a single Map can show as holding megabytes of request objects.
2. Retainer paths. For any object, DevTools walks edges backwards to the roots. That path is the answer to "why wasn't this collected?"

One snapshot shows what is on the heap, but a busy server always holds many objects, so a single snapshot rarely identifies a leak. The method works by comparison. You drive the same workload between snapshots so that legitimate objects reach a steady count, then look for constructors whose instance count grows in proportion to the work done. You then follow one of those instances' retainer paths to the first long-lived object your code owns. That object is the root cause, and the fix is almost always to bound it, remove the entry, or detach the reference when the request finishes.

The rest of this guide follows that sequence: reproduce the growth deterministically, capture snapshots safely, diff them with the three-snapshot technique, read retainer chains, and verify the fix.

Why heap snapshot diffing beats reading code and adding logging

A leak in a long-running API process is a question about references: something reachable from a GC root keeps objects alive that should have been collected. A heap snapshot answers that question directly. Diffing two snapshots shows which constructors grew between them, by how many instances, and how much retained size they hold. Each object's retainer chain then traces the path from the leaking object back to the root that holds it, for example a module-level Map, an event emitter's listener array, or a closure captured by a timer.

Code review and extra logging approach the problem from the other end. Review means guessing which of many plausible patterns (caches, listeners, closures, global registries) is actually growing, across your code and every dependency. Logging process.memoryUsage() confirms that heap growth is happening. It cannot tell you what is growing or why it is still reachable. Both methods are open-ended: you don't know how much time they will take, or whether they will find anything.

Snapshot diffing has a fixed setup cost. You need a load script that drives the suspect endpoints the same way every time. You also need builds that keep meaningful names: unminified code and named functions instead of anonymous handlers, so constructors and closures in the retainer view point to real source. After that, the search is limited to a short list of constructors and one concrete reference path.

Until you find the root cause, you can raise --max-old-space-size or schedule restarts to reduce the impact. Both measures hide the growth that you need to observe. Track them as temporary measures with an owner and a removal date, and do not treat either one as the fix.

How it saves engineering time

The time saved comes from replacing an open-ended search with a bounded one. Code review and logging scale with the size of the codebase and its dependencies. A snapshot diff scales with the number of constructors that grew, which after a clean reproduction is usually a short list.

Specific savings:

• No guessing across dependencies. The retainer chain names the module that holds the objects, including third-party middleware or instrumentation you would not think to review.
• One investigation instead of several speculative deploys. Without evidence, teams often ship a candidate fix, wait hours or days to see whether memory still climbs, and try again. A local reproduction plus snapshot diff confirms the cause before anything ships.
• Reusable setup. The load script, sampler, and capture commands carry over to the next leak. The soak test described later catches regressions in CI, before they reach production and on-call.
• Less ongoing toil. Scheduled restarts and raised heap limits have a recurring cost in alerts, capacity, and incident handling. Removing the root cause ends that cost.

An illustrative estimate, with all inputs assumed rather than measured: assume writing the load script and sampler takes half a day, and each capture-diff-read cycle takes about an hour. If the leak is found on the second cycle, the investigation costs roughly one working day. Compare that with an assumed trial-and-error approach of three speculative fixes, each needing a deploy and a day of observation, plus review time. That comes to at least three days of elapsed time and several interrupted engineers. Your own numbers will differ. The structural point holds: snapshot diffing has a known, mostly fixed cost, while guessing does not.

Reproduce the leak under repeatable load

Before taking any snapshot, make the leak happen on demand. Run the service locally or in staging with production-like configuration, then write a small load script that calls the suspected endpoint with the same method, headers, and payload in a fixed loop. Keep concurrency low and constant, use fixed or seeded input values, and avoid randomness unless you can replay it exactly. Mixed traffic, background jobs, and cache warm-up all add allocations that look like growth but are not the leak.

To confirm the leak, watch the post-GC floor, not raw heap usage. Start the process with --expose-gc. Every few thousand requests, pause the load, call global.gc(), and log process.memoryUsage().heapUsed. A healthy service settles at a stable floor after warm-up. A leaking one shows a floor that climbs in roughly proportion to the number of requests sent. If the increase per thousand requests stays about the same across runs, you have a reproduction you can work with.

This determinism is what makes snapshot diffs useful in the next steps. When each batch performs identical work, objects that survive between snapshots map to the code path you exercised, and their counts scale with the request count instead of varying with noise.

There is a limit to this approach. Some leaks need a specific mix of production requests, a timeout, a client that disconnects mid-stream, or a rejected promise on an error path. A clean staging run means only that the paths you tested do not leak. It does not show that the leak is gone.

Copy// load.mjs: run with `node load.mjs` (Node 18+ for global fetch)
// Start the service with `node --expose-gc server.js`, local/staging only, plus a debug route:
//   app.get('/debug/heap', (req, res) => { global.gc(); res.json({ heapUsed: process.memoryUsage().heapUsed }); });

const BASE = process.env.TARGET ?? '';
const BATCH = 2000;    // requests between post-GC samples
const BATCHES = 10;
const CONCURRENCY = 4; // low and constant, so every batch does the same work
// Fixed payload: no randomness, so each request takes the same code path
const body = JSON.stringify({ customerId: 'cust-0001', items: [{ sku: 'SKU-123', qty: 2 }] });

async function hit() {
  const res = await fetch(`${BASE}/api/orders/quote`, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body,
  });
  await res.arrayBuffer(); // drain the body so unread responses don't distort the run
  if (!res.ok) throw new Error(`unexpected status ${res.status}`); // error paths change what allocates
}

async function runBatch() {
  let remaining = BATCH;
  const worker = async () => { while (remaining-- > 0) await hit(); };
  await Promise.all(Array.from({ length: CONCURRENCY }, worker));
}

// Load is paused while this runs, so the number reflects only retained objects
async function sampleFloor() {
  const res = await fetch(`${BASE}/debug/heap`);
  return (await res.json()).heapUsed;
}

let prev = await sampleFloor();
for (let i = 1; i <= BATCHES; i++) {
  await runBatch();
  const floor = await sampleFloor();
  // A leak shows up as a delta per batch that stays roughly constant
  console.log(`after ${i * BATCH} req: floor=${(floor / 1048576).toFixed(1)} MiB, delta=${((floor - prev) / 1024).toFixed(0)} KiB`);
  prev = floor;
}

A deterministic load script sends one fixed request to the suspected endpoint in equal batches at constant concurrency. After each batch it pauses, triggers a GC through a debug-only route, and logs the post-GC heap floor so you can see whether it rises by a similar amount each batch.

Capturing heap snapshots safely

Once you can reproduce the growth, you need snapshots from the running process. There are three stable ways to take them.

Chrome DevTools over the inspector. Start Node with --inspect, open chrome://inspect, attach to the target, and use the Memory panel to take and compare snapshots. Use this locally or in staging.

v8.writeHeapSnapshot(). Call it from code, such as a test harness or an authenticated admin endpoint. It writes a .heapsnapshot file and returns the filename. This lets you script captures at exact points in a load run.

--heapsnapshot-signal. Start the process with --heapsnapshot-signal=SIGUSR2. Sending that signal later writes a snapshot to the working directory. You add no code and expose no port.

Each method has costs:

• The event loop stops. Writing a snapshot is synchronous, so in-flight requests stall until the file is written. Drain the instance from the load balancer first.
• Memory spikes. The write can need memory comparable to the heap size. A container near its limit can be OOM-killed partway through the write, and you lose the process and the evidence. Leave headroom.
• The inspector port is remote code execution. Anyone who can reach it can run arbitrary code. Bind it to localhost, which is the default, and use an SSH tunnel to reach it. Never use 0.0.0.0.
• Snapshots hold every live string. That includes tokens, credentials, and customer PII. Restrict who can trigger a capture, store the files encrypted, and delete them after analysis.

Copynode --inspect=127.0.0.1:9229 server.js

ssh -N -L 9229:127.0.0.1:9229 deploy@staging-host.example.internal

node -e '
  const v8 = require("v8");
  const file = v8.writeHeapSnapshot();  // blocks the event loop while writing
  console.log("wrote", file);
'

cd /srv/app && node --heapsnapshot-signal=SIGUSR2 server.js &
APP_PID=$!

kill -USR2 "$APP_PID"

chmod 600 /srv/app/*.heapsnapshot

Shows all three capture methods: a loopback-bound inspector reached through an SSH tunnel, a direct v8.writeHeapSnapshot() call, and a SIGUSR2-triggered dump, with snapshot file permissions restricted after capture.

How the three-snapshot technique works

Copywarm-up ──► S1 ──► N requests ──► S2 ──► N requests ──► S3

Warm up first so one-time work finishes before the baseline. Take snapshot 1, run N requests, take snapshot 2, run the same N again, and take snapshot 3. Open snapshot 3 in the Summary view and pick "Objects allocated between Snapshot 1 and Snapshot 2." That filter shows objects created during the first batch that are still alive after the second batch. Those are your leak candidates. The Comparison view (S3 against S2) gives the same signal as # Delta per constructor.

Reading the columns

• Shallow size: memory the object itself holds.
• Retained size: memory that would be freed if the object were collected.
• Distance: number of hops from a GC root.
• Count delta: net change in instances between the two snapshots.

Sort by count delta to find types that grow with request volume. If 2,000 requests produced roughly 2,000 new instances, you probably have a per-request leak. Sort by retained size to find the object holding most of that memory. Plain strings and closures usually rank high, but they are often symptoms. Ask what retains them.

False positives

Some growth is expected:

• lazily initialized modules
• JIT-compiled code
• connection pools filling to their limit
• caches with a fixed size limit

All of these grow early and then stop. If you skip warm-up, they land between S1 and S2 and look like leaks. If N is too small, a real leak produces a delta too small to distinguish from this noise. A real leak keeps growing from S2 to S3. Bounded growth does not.

Copy ┌───────────┐      ┌────────────┐      ┌────────────┐
 │  warm-up  │─►S1─►│ N requests │─►S2─►│ N requests │─►S3
 │ (no snap) │      │  batch A   │      │  batch B   │
 └───────────┘      └────────────┘      └────────────┘
  lazy init,        allocations         same load:
  JIT, pools,       here = leak         survivors
  caches fill       candidates          keep growing

  ▲ heap
  │                                                     ╱ leak: S2→S3 grows
  │                                                ╱
  │                                           ╱
  │                                      ╱
  │                                 ╱
  │                            ╱
  │                       ╱
  │    ╱─────────────╱─────────────────────────────────── bounded: flat
  │   ╱
  │  ╱
  │ ╱
  └─────────────┬───────────────────┬───────────────────┬──► time
     warm-up    S1                  S2                  S3

  ┌─────────────────────────────────────────────────────────────┐
  │ Summary view of S3, filter: allocated between S1 and S2     │
  │ Comparison S3 vs S2 (# Delta) ─► alive in S3 = suspects     │
  │ Sort by count delta (growth) and retained size (holder)     │
  └─────────────────────────────────────────────────────────────┘

Warm-up absorbs one-time growth before S1, so objects allocated between S1 and S2 that are still alive in S3, and still growing, separate a real leak from bounded growth such as pools and fixed-size caches.

Following retainer chains to common leak patterns

Open the comparison between S2 and S3, sort by size delta, and expand a constructor whose count grew by roughly N. Select one instance and read the Retainers panel bottom-up. Each row shows the property or slot holding the object. Keep expanding the shortest path until you reach something your code owns, such as a module-level Map, an emitter's listener array, a timer, or a cache. The objects in between are symptoms. The root cause is the first long-lived owner.

Names make this possible. A named function or class shows up as itself in the chain. An anonymous arrow function's captured variables show up only as a generic closure context, which tells you nothing about its source. Minified builds rename everything, so take snapshots from an unminified build with named handlers.

The code example that follows shows the listener case before and after the fix.

Verifying the fix, production evidence, and alternative tools

Rerun the identical load script and the same three-snapshot sequence against the patched build. The fix holds if the candidate constructors from the leak (closures, listener arrays, Map entries) show near-zero net growth between snapshots two and three, and if heapUsed measured after a forced GC levels off instead of climbing with each load cycle. A lower peak alone proves nothing. The floor is what matters, because unreferenced objects can sit on the heap until the next major collection.

To stop the leak from returning, add a soak test. Run it with --expose-gc in the test job only, so production code never calls global.gc(). The test drives a few thousand requests, forces GC, reads process.memoryUsage().heapUsed, repeats the cycle, and asserts that the final value stays within a fixed bound of the first post-warm-up value. Measure after warm-up, because JIT and cache fill-up cause legitimate early growth.

In production, remove one instance from load balancer rotation before taking a snapshot. Taking a snapshot pauses the process and can roughly double its memory use. Alert on a rising post-GC floor across hours, not on raw usage, since raw usage naturally oscillates with V8's GC cycle.

If RSS rises while heapUsed stays flat, the problem is outside the JavaScript heap, and snapshots will not show it. Check external and arrayBuffers first. If those are also flat, investigate native addons and the system memory allocator with OS-level tools.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to node.js memory leak debugging, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.