Is it actually a memory leak, or just a large heap?
A JavaScript heap leak means objects stay reachable from a GC root after the code that needed them has finished, so the garbage collector cannot reclaim them. The sign to look for is a rising floor: heapUsed measured right after a full GC keeps climbing across repeated, identical units of work. A heap that is large but returns to the same baseline after each GC is not leaking. It is just big, and a heap snapshot will mostly show you legitimate caches.
rss alone can mislead you. It counts everything the process holds, including Buffer backing stores, native addon allocations, and pages the allocator has not returned to the OS. process.memoryUsage() reports heapUsed, external (memory bound to JS objects but allocated outside the V8 heap), arrayBuffers (the ArrayBuffer and Buffer portion of external), and rss. The script below logs all four at a fixed interval. If you start Node with --expose-gc, it calls global.gc() before each sample so the readings show post-GC floors and not sawtooth peaks.
| Symptom | Likely cause | Right tool |
|---|---|---|
Post-GC heapUsed floor rises steadily |
Retained JS objects | Heap snapshot diffing |
heapUsed high but flat across cycles |
Large working set or caches | Capacity review; snapshots optional |
external/arrayBuffers rise, heapUsed flat |
Buffers held or not released | Audit Buffer and stream lifetimes |
rss rises, all V8 counters flat |
Native addon leak or allocator fragmentation | Native memory profiler (Valgrind massif, heaptrack); snapshots won't show it |
// memwatch.js: log post-GC memory floors at a fixed interval.
// Run: node --expose-gc memwatch.js (or require it early in your app)
const INTERVAL_MS = 10_000; // sample every 10s; align with your unit of work
const toMB = (n) => (n / 1024 / 1024).toFixed(1);
function sample() {
// Force a full GC when available so heapUsed reflects the floor,
// not a point on the allocation sawtooth.
if (typeof global.gc === 'function') global.gc();
const m = process.memoryUsage();
// One line per sample, easy to grep or plot later.
console.log(JSON.stringify({
t: new Date().toISOString(),
gcForced: typeof global.gc === 'function',
heapUsedMB: toMB(m.heapUsed), // live JS objects on the V8 heap
externalMB: toMB(m.external), // C++ memory bound to JS objects
arrayBuffersMB: toMB(m.arrayBuffers), // Buffer/ArrayBuffer share of external
rssMB: toMB(m.rss), // everything the OS has given the process
}));
}
if (typeof global.gc !== 'function') {
console.warn('Start with --expose-gc for post-GC readings; values include garbage.');
}
sample();
// unref() so the sampler never keeps an otherwise finished process alive.
setInterval(sample, INTERVAL_MS).unref();
A minimal sampler that forces a GC (when started with --expose-gc) and logs heapUsed, external, arrayBuffers, and rss as JSON lines so you can tell a rising post-GC heap floor from external or native memory growth.
Why does a single heap snapshot mislead you in a monolith?
Once you have confirmed that V8 heap usage really is growing, the obvious next step is to take a heap snapshot and sort by retained size. In a monolith, this rarely gets you far. A single snapshot shows everything the process holds at one moment, and most of it is legitimate. The top entries are usually compiled code, the module cache, framework routing tables, ORM metadata, connection pools, template caches, and large strings from configuration or bundled assets. Lazily loaded modules make this worse: a route hit for the first time pulls in a dependency tree, and that allocation looks the same as a leak in a static picture.
The problem is that one snapshot has no baseline. A 40 MB Map might be a correctly sized LRU cache or a session store that never evicts. You can't tell which without seeing how it changes over time.
The method this tutorial uses addresses that directly:
- Drive the process with a controlled workload that repeats the same operations.
- Take snapshots at points where the process should be in the same steady state.
- Diff the snapshots so that only objects that survived and accumulated remain.
- Follow their retainer paths until you reach a module, closure, or data structure in your own code.
This matters most for backend and platform engineers who own long-running monolith processes, where the heap contains code from dozens of teams. Reading the largest objects one by one means investigating framework internals you don't own and can't change. Diffing cuts the candidate list down to the objects that grow, so the investigation starts in your code and not in a dependency.
How does heap snapshot diffing work?
A V8 heap snapshot is a serialized graph. Each node is a heap object with a type, a constructor or class name, a shallow size, and an object ID. Each edge is a reference from one object to another, such as a property, an array element, a closure's captured context, or an internal pointer. The roots of the graph are the GC roots: global objects, the module system, active handles, the stack, and V8 internals.
Within a single process, V8 assigns each heap object an ID that stays the same across every snapshot taken while that object is alive. Diffing depends on this. When DevTools compares two snapshots from the same process, it can tell which objects already existed in the earlier snapshot, which are new, and which were collected. Snapshots from different processes, or from the same service after a restart, have unrelated IDs. They can only be compared by aggregate counts per constructor, which is much weaker evidence.
From the graph, DevTools computes a dominator tree. Retained size comes from that tree, and retainer paths come from walking edges backward from an object toward a root. The workflow in the rest of this article combines the two. The diff narrows the heap to objects that keep accumulating under a fixed workload. The retainer path then tells you which reference keeps them alive. That reference is what you change.
Who should use heap snapshot diffing?
This technique fits engineers who are responsible for long-running Node.js processes whose memory keeps growing:
- Backend engineers on a shared monolith. They need to find which of many modules is retaining memory without first reading every team's code.
- Platform engineers and SREs. They handle OOM kills, container restarts, or "JavaScript heap out of memory" crashes and need evidence that points to an owner before handing off the problem.
- Teams running queue consumers, WebSocket servers, or other long-lived workers. Small per-message leaks in these processes add up over days of uptime.
It requires a staging or local environment where the suspect path can be reproduced, permission to run Node with flags like --expose-gc or --inspect there, and a workstation that can load multi-gigabyte snapshot files. Engineers working mainly on short-lived processes, such as CLI tools, build scripts, or functions that exit after each invocation, rarely need it. In those cases the process exit frees the memory before a leak can matter.
When should you use heap snapshot diffing?
Use it when the evidence points to retained JavaScript objects and you can repeat the work that causes the growth:
- The post-GC
heapUsedfloor rises across repeated, identical units of work, as described in the first section. - Processes are being OOM-killed or restarted, and the memory curve before each crash shows a steady rise that tracks traffic or job volume rather than a single spike.
- Growth started after a specific deploy, and you need to find which change is retaining memory.
- Metrics implicate a specific route, job, or tenant pattern that you can drive in isolation with a harness.
- A single snapshot showed only large, plausible containers such as Maps, caches, and pools, and you can't tell which of them is growing.
It is also the right tool to verify a fix. Rerunning the same harness and snapshots, and confirming that the growing constructor no longer grows, is stronger evidence than watching a production graph for a week.
When should you avoid heap snapshot diffing?
Choose a different approach in these cases:
heapUsedis flat butrss,external, orarrayBuffersgrows. The memory is outside the V8 object graph, so snapshots won't show the cause. Audit Buffer and stream lifetimes, or use native memory tooling, as shown in the first table.- The heap is large but stable. If the post-GC floor returns to the same level each cycle, there is nothing to diff. This is a capacity or cache-sizing question.
- The only available process is serving traffic or is close to its memory limit. Capture blocks the event loop and can temporarily need memory comparable to the heap size. It can cause the outage you are investigating. If you can't drain a replica or reproduce the problem elsewhere, start with the sampler and metrics.
- You cannot reproduce the growth under a controlled workload. Snapshots taken during mixed, uncontrolled traffic produce noisy diffs. The allocation-tracking alternatives near the end of this article, or narrowing down the cause with production metrics first, may get you further.
- The data in memory cannot leave its environment. Snapshots contain every live string, including tokens and personal data. If policy forbids copying that data to a workstation, analyze it inside the controlled environment or reproduce the leak with scrubbed data.
- The growth is a bounded cache still filling up. Check the configured limits first. A cache that plateaus at its maximum is not a leak.
How do you build a reproducible workload for heap profiling?
Diffing only works if the growth between snapshots comes from one known activity. In a monolith serving dozens of routes, background jobs, and scheduled tasks, unrelated traffic adds noise to every diff. Narrow the scope first. Pick the single request path or job your metrics implicate, such as the endpoint whose traffic correlates with heap growth or the queue consumer that runs before memory climbs. Drive only that path, in staging or locally, with background schedulers disabled where you can.
A load tool is fine for HTTP routes. A plain loop script that calls the handler or enqueues the job is often better, because it gives you exact control over how many times the code runs. Use realistic payloads: a leak keyed on user ID or tenant will not appear if every request sends identical input.
Warm up before the first capture. The first few hundred iterations populate caches, trigger lazy require calls, and let the JIT compile hot functions. All of that allocates memory that stays live but is bounded. If your baseline snapshot comes before warm-up, those one-time allocations show up as growth and look like a leak.
After warm-up, run a fixed iteration count between captures, for example 1,000 requests each time. With equal workloads, a real leak grows by a consistent amount per interval.
This setup has a cost. Staging cannot reproduce production tenant data, traffic mix, or weeks of uptime, and a synthetic workload only exercises the code paths you chose to drive. You will find leaks along the path you exercise and miss leaks elsewhere.
// Run: node --expose-gc scripts/leak-harness.js
// Drives one suspected path in-process so no other routes, jobs or schedulers add noise.
const v8 = require('node:v8');
const { buildInvoicePreview } = require('../src/billing/invoicePreview'); // suspected path
const WARMUP = 500; // fills caches, triggers lazy requires, lets the JIT compile hot code
const ITERATIONS = 1000; // fixed workload between every pair of snapshots
const ROUNDS = 3;
// Vary tenant and user so leaks keyed on those values can appear
function makePayload(i) {
return {
tenantId: `tenant-${i % 50}`,
userId: `user-${i}`,
lineItems: [{ sku: 'SKU-1001', qty: (i % 5) + 1 }],
};
}
async function drive(count, offset) {
for (let i = 0; i < count; i++) {
await buildInvoicePreview(makePayload(offset + i));
}
}
function capture(label) {
global.gc(); // collect short-lived garbage so the numbers reflect retained memory
const mb = (process.memoryUsage().heapUsed / 1e6).toFixed(1);
const file = v8.writeHeapSnapshot(`heap-${label}.heapsnapshot`);
console.log(`${label}: heapUsed=${mb} MB -> ${file}`);
}
(async () => {
await drive(WARMUP, 0);
capture('0-baseline'); // baseline taken only after warm-up
let offset = WARMUP;
for (let r = 1; r <= ROUNDS; r++) {
await drive(ITERATIONS, offset);
offset += ITERATIONS;
capture(`${r}-after-${r * ITERATIONS}`); // equal work per interval: a real leak grows by a similar amount each round
}
})();
A loop harness that warms up a single suspected code path, then runs a fixed number of iterations with varied tenant and user payloads before each heap snapshot.
How do you capture heap snapshots safely?
Whichever workload you settle on, capturing the snapshot carries its own risks. V8 serializes the whole heap synchronously, so the event loop stops for the entire capture and health checks and in-flight requests stall with it. The process can also temporarily need memory comparable to the current heap size. On a process already close to its limit, that extra demand can cause the OOM you were trying to diagnose. Take snapshots on a replica you have removed from the load balancer, never on one that is still serving traffic.
| Method | How it's triggered | Best for | Main risk |
|---|---|---|---|
--inspect + Chrome DevTools |
Attach debugger, click "Take snapshot" | Local and staging repro loops | Exposes a debug port; unsafe to open in production |
--heapsnapshot-signal=SIGUSR2 |
kill -USR2 <pid> |
Ops-driven capture on a drained replica | Needs shell access; writes to the process's working directory |
v8.writeHeapSnapshot() |
Admin HTTP route in your code | Repeatable captures timed with the workload | Becomes a denial-of-service vector if left unguarded |
If you expose v8.writeHeapSnapshot() through an admin route, apply these guards:
- Require authentication on the route.
- Allow one capture per process at a time.
- Enforce a cooldown of several minutes between captures.
- Write the file to a dedicated volume that has room for it.
- Return only the file path, not the file contents.
Monolith snapshots are often several gigabytes. DevTools may load them slowly or fail to load them at all, so compare them on a workstation with plenty of RAM.
Treat every snapshot as a credentials dump. It contains every live string in memory, including session tokens, API keys, and customer personal data. Store snapshots in an access-controlled location, record who downloads them, and delete them automatically once the investigation is finished.
const http = require('node:http');
const v8 = require('node:v8');
const path = require('node:path');
const crypto = require('node:crypto');
const TOKEN = process.env.HEAP_ADMIN_TOKEN; // injected from your secret store
const DIR = '/var/heapdumps'; // dedicated, access-controlled volume sized for multi-GB files
const COOLDOWN_MS = 10 * 60 * 1000; // at most one capture per 10 minutes per process
let lastCapture = 0;
let capturing = false;
function authorized(req) {
if (!TOKEN) return false; // no token configured means the route stays disabled
const given = Buffer.from(req.headers['authorization'] || '');
const expected = Buffer.from(`Bearer ${TOKEN}`);
return given.length === expected.length && crypto.timingSafeEqual(given, expected);
}
// Separate admin listener on loopback, never on the public port.
http.createServer((req, res) => {
if (req.method !== 'POST' || req.url !== '/admin/heap-snapshot') return res.writeHead(404).end();
if (!authorized(req)) return res.writeHead(401).end();
const now = Date.now();
if (capturing || now - lastCapture < COOLDOWN_MS) return res.writeHead(429).end();
capturing = true;
lastCapture = now;
try {
// Synchronous: the event loop is blocked until the file is written.
// Trigger only on a replica already removed from the load balancer.
const file = v8.writeHeapSnapshot(path.join(DIR, `heap-${process.pid}-${now}.heapsnapshot`));
res.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify({ file })); // path only, never contents
} catch (err) {
res.writeHead(500).end();
} finally {
capturing = false;
}
}).listen(9230, '127.0.0.1');
An admin-only HTTP trigger for v8.writeHeapSnapshot() that requires a bearer token, allows one capture at a time with a cooldown, writes to a dedicated volume, and returns only the file path.
How does the three-snapshot technique work?
Take three snapshots against the steady workload. Snapshot 1 comes after warm-up, so module loading, JIT compilation, connection pools, and lazily filled caches are already in the baseline. Snapshot 2 comes after N iterations, and snapshot 3 after another N.
start ──warm-up──▶ [S1] ──N iterations──▶ [S2] ──N iterations──▶ [S3]
baseline first growth confirms growth
In the DevTools Memory panel, open S3's Summary view and choose "Objects allocated between Snapshot 1 and Snapshot 2." These objects were created during the first batch and still survive after the second batch. One-off allocations drop out of this view. Next, open the Comparison view for S3 against S2 and look at # Delta. A real leak grows in proportion to the workload: if 1,000 requests leave about 1,000 extra objects in each interval, you have a strong candidate.
Each object has two sizes. Shallow size is the memory the object itself holds. Retained size is the memory that would be freed if that object were collected. An object dominates another when every path from a GC root to the second object passes through it. Retained size is the shallow size plus everything the object dominates. A single Map entry is small, but the Map holding 50,000 entries dominates all of them. Sorting by retained size therefore puts containers such as Maps, arrays, and caches at the top.
Select a leaked object and read the Retainers panel. It shows the chain that keeps the object alive, all the way back to a root:
(GC root) ─▶ system / Context ─▶ module closure
─▶ sessionCache (Map) ─▶ entry ─▶ RequestContext ← leaked
SNAPSHOT TIMELINE (steady workload)
┌───────┐ warm-up ┌──────┐ N iter ┌──────┐ N iter ┌──────┐
│ start │────────▶│ S1 │───────▶│ S2 │───────▶│ S3 │
└───────┘ └──────┘ └──────┘ └──────┘
baseline first growth confirms growth
│ │
└──▶ S3 Summary: objects allocated between S1 and S2
S3 vs S2 Comparison: # Delta tracks workload
RETAINER CHAIN (Retainers panel walks this from the leak back to root)
┌───────────┐
│ (GC root) │
└─────┬─────┘
▼
┌──────────────────┐
│ system / Context │
└─────┬────────────┘
▼
┌────────────────┐
│ module closure │
└─────┬──────────┘
▼
┌────────────────────┐ dominator: small shallow size,
│ sessionCache (Map) │ retained = Map + all entries,
└─────┬──────────────┘ so it sorts to the top by retained size
▼
┌───────┐ ┌────────────────┐
│ entry │──▶│ RequestContext │◀── leaked: about +N per interval
└───────┘ └────────────────┘
Snapshots S1–S3 bracket two equal batches of work; objects allocated between S1 and S2 that are still alive in S3 lead through the Retainers panel to a GC root, with the sessionCache Map as the dominator.
Which retainer paths point to code you own?
In a monolith, most of that chain is noise. On a first pass, skip rows labelled (system), (compiled code) and (array), and skip framework internals such as router layers or ORM metadata. They usually retain your objects; they rarely own them. Type your own class and constructor names into the Summary view's class filter instead. TenantContext, InvoiceDraft or OrderDto growing between snapshots is a stronger signal than any amount of (string).
Then walk each retainer chain upward until you reach something your team wrote and can change: a module-level variable, a singleton, an EventEmitter, or a closure. That node is where the fix goes. The objects further down the chain are only what it happens to hold.
| Pattern | Retainer signature | Fix |
|---|---|---|
| Memoization keyed by request data | module closure → Map → entries keyed by IDs or query strings |
Use an LRU or TTL bound, or a key with low cardinality |
| Per-request listener never removed | EventEmitter → _events → array of closures |
Call off() in a finally block, or use once() |
Closure capturing req/res |
(closure) → context → IncomingMessage |
Capture only the fields you need |
| Uncleared interval | Timeout → _onTimeout → closure → your object |
Call clearInterval on shutdown or completion |
| Per-tenant registry | singleton → Map keyed by tenant → services |
Evict when a tenant goes idle |
Two caveats apply. A bounded, intentional cache grows until it reaches its limit and then plateaus, so check its configured limit before calling it a leak. A real leak may also have no single large retainer. Thousands of small objects spread along one chain can add up to the growth while every individual node looks harmless, so sort by retained size and by count.
What are the alternatives when snapshots are ambiguous, and how do you prevent regressions?
When retainer chains end in generic framework code such as a router, an event emitter or a promise queue, use allocation instrumentation on timeline in DevTools. It records the call stack for each allocation that is still alive, so you can see which handler created the objects, not just what holds them. It is expensive, so run it only in staging. The sampling heap profiler costs much less and works for longer runs, but because it samples, it can miss small, frequent allocations.
If RSS grows while heapUsed stays flat, heap snapshots will not show the leak. Check external and arrayBuffers in process.memoryUsage(), then use a native allocator profiler on the process.
| Approach | Finds | Cost | Fixes leak? |
|---|---|---|---|
| Allocation timeline | Allocating stack | High overhead | Diagnoses |
| Sampling profiler | Hot allocation sites | Low overhead | Diagnoses |
| Native tooling | Off-heap growth | Setup effort | Diagnoses |
| Higher heap limit or restarts | Nothing | Low | No, hides it |
Raising the heap limit or scheduling restarts only delays the crash. The leak keeps growing, and the tooling that would show it is gone. To verify a fix, rerun the same workload and the same three snapshots, and confirm the growing constructor is gone.
To prevent regressions, bound every cache with eviction, remove listeners when their owner is torn down, and alert on the trend of heap size after GC rather than on absolute size.
Illustrative estimate (assumed inputs): three days to set up a staging workload with scrubbed data, against four leak incidents a year at one engineer-day each. The setup pays for itself within the first year.
How can heap snapshot diffing save engineering time?
Most of the time spent on a monolith memory leak goes to search, not to the fix. The fix is usually small: add an eviction bound, remove a listener, or stop capturing req in a closure. Diffing shortens the search in four ways:
- It narrows the suspects. A single snapshot of a monolith puts thousands of legitimate objects in front of you. A three-snapshot diff under a fixed workload leaves only the constructors that grow with the work. You review a short list, not the whole heap.
- It routes the problem to the right owner. The retainer path ends at a specific module-level variable, singleton, emitter, or closure. Its location in the codebase tells you which team owns it. You avoid cross-team code reviews and debates about whose dependency is leaking.
- It replaces recurring mitigation with one fix. Scheduled restarts and higher heap limits cost on-call time and investigation effort every time the process degrades. Finding the retaining reference ends that cycle.
- It makes verification quick and repeatable. The same harness and the same snapshot sequence confirm a fix in a single run. You don't have to wait days to see whether the production memory curve flattens. Once the harness exists, later investigations on that code path start from a working setup.
The return depends on how often leaks occur and how long they currently take to track down. The illustrative estimate above shows how to reason about it with your own assumed inputs. Weigh the one-time cost of the harness and staging data against the engineer-days your team currently spends on each leak, including the time spent on restarts and mitigation.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to node.js memory leak heap snapshot, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.