What is Node.js event loop lag and why does one slow call block every request?

Event loop lag is the delay between the moment a callback becomes ready to run and the moment the JavaScript thread actually runs it. Ready callbacks include a socket with request bytes, a resolved database query, and an expired timer. Node.js runs all your JavaScript on one thread per process. When a function runs synchronously, nothing else in that process's JavaScript can run until it returns. Every other ready callback waits in the queue: in-flight API requests, database responses and the /healthz handler your load balancer polls.

time (ms)    0         50        100       150       200   210   220
JS thread    [==== /export: sync JSON.stringify, 200 ms ====][A ][H ][B ]
ready at          ^A(30)          ^H(110)        ^B(160)
lag               A: 170 ms       H: 100 ms      B: 50 ms

The diagram explains why the handler that looks slow is often innocent. Request A, a simple lookup, shows 170 ms of latency in your traces, but it did almost no work. The cost lands on whichever requests are waiting when the blocking call runs, not on the code that caused it. If the health check (H) times out under a heavier version of this pattern, the orchestrator may restart a process that is otherwise healthy.

Illustrative estimate. Assume these inputs:

  • 500 requests/s on one process.
  • 1 in 250 requests triggers a 200 ms synchronous operation.
  • All other handlers take about 2 ms.

That gives two blocks per second, so the thread is blocked about 40% of the time. Roughly 40% of all requests arrive during a block and wait an extra 0–200 ms, about 100 ms on average. Median latency stays low, but p90 and p99 climb toward 200 ms or more for every endpoint on the process. The backlog that builds up during each block makes the tail longer still.

 +-----------------+  +-----------------+  +-----------------+
 | A: GET /user    |  | H: GET /healthz |  | B: GET /orders  |
 | (2 ms of work)  |  | (load balancer) |  | (2 ms of work)  |
 +--------+--------+  +--------+--------+  +--------+--------+
          |                    |                    |
          v                    v                    v
 +------------------------------------------------------------+
 | Node.js process                                            |
 |                                                            |
 | kernel/libuv: sockets, DB replies, timers become ready     |
 |                             |                              |
 |                             v                              |
 |  +------------------------------------------------------+  |
 |  | queue of ready callbacks                             |  |
 |  | [A @30ms] -> [H @110ms] -> [B @160ms]                |  |
 |  +--------------------------+---------------------------+  |
 |                             | waits until thread is free   |
 |                             v                              |
 |  +------------------------------------------------------+  |
 |  | single JavaScript thread                             |  |
 |  | 0-200 ms BUSY: /export sync JSON.stringify           |  |
 |  | then A (lag 170) -> H (lag 100) -> B (lag 50)        |  |
 |  +------------------------------------------------------+  |
 +------------------------------------------------------------+

Requests A, H (the health check) and B become ready during a 200 ms synchronous /export call and wait in the callback queue until the single JavaScript thread is free. Each one absorbs lag (in ms) that it did not cause.

Why don't CPU and memory dashboards show event loop lag?

A Node.js process runs JavaScript on a single main thread, so a blocked loop pins one core while the others sit mostly idle. Host-level CPU dashboards usually average across cores. On an 8-core machine, one core at 100% and the rest near zero reads as roughly 12–13% utilization (illustrative estimate). That number looks healthy, and nobody gets paged. Per-process CPU fares a little better, but it is usually sampled over intervals long enough to smooth a 150 ms block into a mild bump. Memory is unrelated: a synchronous loop over a parsed payload can block for seconds without allocating much.

The symptom shows up elsewhere. Lightweight handlers like health checks, cache hits, and auth lookups get slower even though their own code never changed, because they wait behind whatever holds the thread. If your latency budget matters, treat event loop delay as a first-class production metric alongside request latency, not as a debugging curiosity. It is the only signal that points directly at "the thread was busy."

It is not always a sign of bad application code. Garbage collection pauses also stop the main thread and appear as loop delay. Before rewriting handlers, check GC activity, either through perf_hooks GC entries or by running with --trace-gc. If lag spikes line up with major collections, the fix is allocation pressure or heap sizing.

Adding instances or cores lowers the share of traffic that hits a blocked process. It does not remove the block, and it raises the cost of hiding it.

Who should monitor event loop lag?

Anyone who runs a long-lived Node.js process that serves requests and has a latency expectation should track it. That includes:

  • Backend teams running HTTP, gRPC or WebSocket services in Node.js. Every connection on a process shares one JavaScript thread, so one handler's synchronous work becomes every client's latency.
  • Teams whose request sizes vary with the customer or the input. Multi-tenant APIs, bulk import/export endpoints, GraphQL servers and backend-for-frontend layers that reshape large JSON payloads are where blocking work tends to grow unnoticed.
  • SREs and on-call engineers who own latency targets or health checks. Lag is the metric that explains p99 regressions and probe timeouts that request traces and host CPU graphs do not.
  • Engineers reviewing changes that add CPU work to request paths, such as hashing, compression, templating, validation or parsing.

It matters less for short-lived CLI tools and batch scripts where only total runtime counts. Even there, track it if the same process also serves a health endpoint, a metrics endpoint or a queue heartbeat, because those wait behind the batch work like any other callback.

When should you use event loop lag monitoring?

For latency-sensitive services, run it continuously in production, not only during incidents. The histogram is cheap to keep enabled, and a blocking pattern often appears only with real data sizes. Lean on it specifically in these situations:

  • p99 latency rises across unrelated endpoints while database, cache and downstream latency stay flat. That pattern points at the process, not its dependencies.
  • Health checks or liveness probes time out intermittently, or the orchestrator restarts instances that show no crash or memory problem.
  • After deploying endpoints that process variable-size input, or after a large customer onboards. Lag that grows with tenant data shows up weeks later, not at deploy time.
  • During load tests and capacity planning, to see whether a process saturates its thread before it saturates anything else.
  • Before and after a fix, to confirm that moving work to async APIs, chunks or workers actually reduced delay.

Lag is the wrong tool when requests are slow because they wait on I/O. A slow database query leaves the thread idle and loop delay flat. In that case, look at request traces and downstream latency instead. Likewise, async crypto/zlib work that is queued on a saturated libuv threadpool raises request latency without raising loop delay, as described in the fixes section below.

How does the event loop work, and how is lag measured?

Node's event loop is provided by libuv. Each iteration moves through a fixed sequence of phases: timers (expired setTimeout/setInterval callbacks), pending callbacks, an internal idle/prepare step, poll (waiting for and running I/O callbacks such as incoming socket data), check (setImmediate callbacks) and close callbacks. After each callback, Node drains the process.nextTick queue and the promise microtask queue before moving on. When there is nothing to do, the loop sits in the poll phase waiting for the operating system to report new I/O.

Every callback runs to completion. Nothing preempts a JavaScript function, so if one callback runs for 200 ms, every timer that expires and every socket that becomes readable during those 200 ms waits until it returns and the loop reaches the relevant phase. That waiting time is the lag.

The two built-in measurements observe this from different angles:

  • monitorEventLoopDelay() schedules a recurring libuv timer at the resolution interval. Each time the timer fires, it records how much later it ran than it was scheduled to, in nanoseconds, into a histogram. If the thread was free, the delay is near zero. If a synchronous call held the thread, the delay is roughly how long the timer had to wait. Because it is a sample per interval, the histogram gives you percentiles rather than a single number.
  • performance.eventLoopUtilization() uses libuv's accounting of how long the loop spent idle, waiting for events in the poll phase, versus how long it spent active running callbacks. Utilization is the active share of elapsed time, from 0 to 1. It does not tell you how long any callback waited. It tells you how busy the thread was.

Delay answers "did ready work have to wait?" and utilization answers "how much of the time was the thread doing something?" Reading both together is what separates a busy but responsive loop from a blocked one, as the next section shows.

How do you measure event loop lag with perf_hooks?

Node ships a purpose-built sampler for event loop lag. perf_hooks.monitorEventLoopDelay() returns a histogram. Call enable() once at startup. On a fixed interval, read percentile(50), percentile(99), max and mean, then call reset() so each read covers only that window. All values are in nanoseconds, so divide by 1e6 before exporting milliseconds.

The resolution option sets the sampling timer period in milliseconds and defaults to 10. A finer resolution catches shorter stalls but fires more timers, which adds a small amount of overhead to the loop you are measuring. A coarser resolution is cheaper but can miss stalls shorter than the period. For API services, the default is usually adequate, because a stall short enough to fall between samples rarely affects request latency.

Treat max as a clue, not a signal. A single garbage collection pause or one cold-path JIT compilation can set it, and one outlier per window makes it jump around. p99 over a window tracks what a meaningful fraction of requests actually waited.

Delay alone cannot distinguish a saturated loop from a stuck one. performance.eventLoopUtilization() reports the fraction of time the loop spent active rather than idle. Pass the previous reading to get a per-window delta. High utilization with low p99 means the process is busy but responsive. High utilization with high p99 means it is blocked.

Technique Granularity Distribution Separates busy from blocked
setTimeout drift One sample per timer tick Single value per tick No
monitorEventLoopDelay Set by resolution Percentiles, mean, max No
eventLoopUtilization Per read window Ratio Yes, paired with delay

The setTimeout-drift approach is coarser for three reasons. It produces one sample per tick, so a stall shorter than the gap between ticks can go unrecorded. The timer itself runs on the loop it is measuring, so its own scheduling distorts the result. It also yields no distribution, which leaves you without percentiles to alert on.

const { monitorEventLoopDelay, performance } = require('node:perf_hooks');

// Sample loop delay every 20 ms (default is 10). Coarser = cheaper, but stalls
// shorter than the period can slip between samples.
const histogram = monitorEventLoopDelay({ resolution: 20 });
histogram.enable();

const REPORT_INTERVAL_MS = 10_000;
const NS_PER_MS = 1e6;

// Baseline for per-window utilization deltas.
let lastElu = performance.eventLoopUtilization();

const timer = setInterval(() => {
  // Passing the previous reading returns the delta for this window only.
  const elu = performance.eventLoopUtilization(lastElu);
  lastElu = performance.eventLoopUtilization();

  // Histogram values are nanoseconds; convert before exporting.
  const stats = {
    p50Ms: histogram.percentile(50) / NS_PER_MS,
    p99Ms: histogram.percentile(99) / NS_PER_MS,
    maxMs: histogram.max / NS_PER_MS,   // noisy: one GC pause can set it
    meanMs: histogram.mean / NS_PER_MS,
    utilization: elu.utilization,       // 0..1, share of time the loop was active
  };

  // High utilization + low p99: busy but responsive.
  // High utilization + high p99: blocked; requests are waiting.
  console.log(JSON.stringify(stats));

  // Start a fresh window so the next read covers only the next interval.
  histogram.reset();
}, REPORT_INTERVAL_MS);

// Do not keep the process alive just for metrics.
timer.unref();

Reads p50/p99/max/mean event loop delay in milliseconds every 10 seconds, resets the histogram after each read, and pairs it with per-window event loop utilization to tell a busy loop apart from a blocked one.

How should you alert on event loop lag in production?

Export lag as a distribution. The perf_hooks histogram returned by monitorEventLoopDelay() exposes percentiles directly. On each metrics scrape, publish p50, p99 and max, then reset the histogram so every sample covers one window rather than the whole process lifetime. The values are in nanoseconds, so convert them to milliseconds before export.

Alert on sustained p99, not max

A single max spike is often noise. A large garbage collection, a cold require at startup, or one oversized payload can produce it. Alert when p99 stays above your threshold for several consecutive windows. A sustained p99 means the loop is repeatedly unavailable, and that is the condition your users actually experience. Keep max on the dashboard for context, but don't page anyone on it.

Correlate lag with request logs

When an alert fires, take the time window and join it against request logs. Look for routes that were in flight during the lag and whose own latency jumped. If one route appears in every bad window, it is your first suspect. Include a payload size in logs where you can, because blocking code is often fine on small inputs and pathological on large ones.

Keep liveness probes from causing restart storms

A liveness endpoint is served by the same loop it is meant to check. Under heavy load, the probe response waits behind blocking work, times out, and the orchestrator kills a process that was slow but healthy. Its traffic moves to the remaining instances, they lag in turn, and the restarts cascade.

To prevent this:

  • Use readiness to shed traffic from a lagging instance.
  • Reserve liveness for a loop that has made no progress at all.
  • Give liveness a timeout and failure count well above your lag alert threshold, so the alert always fires first.

Lag metrics also save investigation time. They point at the process itself, so you can skip chasing the database, the network and the load balancer.

What blocking code commonly causes lag in API services?

Most lag in API services traces back to a short list of patterns. Each one does synchronous work on the main thread, and that work grows with input size. A request that is fast in testing can stall the process once a larger customer or a hostile input arrives.

Culprit Typical trigger Symptom
Large JSON.parse / JSON.stringify Bulk endpoints, cached blobs, verbose logging of full payloads Lag spikes on specific routes, scaling with body or response size
Synchronous fs, crypto, zlib readFileSync in handlers, pbkdf2Sync for logins, gzipSync on responses Steady lag that rises with traffic to one endpoint
Catastrophic regex backtracking Nested quantifiers such as (a+)+ applied to user input A single request freezes the process for seconds; easy to trigger deliberately
Large in-memory sorts or loops Queries without LIMIT, client-side filtering or joining of result sets Lag grows as tenant data grows, often weeks after a deploy
Per-request template or schema work Compiling templates or validators inside the handler instead of once at startup Uniform overhead on every request; p99 drifts upward under load

The regex case is the most dangerous. V8's engine backtracks, so a crafted string can force exponential work, and a single request is enough to cause it. Async crypto and zlib calls avoid blocking the loop, but they queue on the libuv threadpool, which defaults to four threads. Moving that work there shifts the contention elsewhere and does not remove it. Before profiling, search your codebase for Sync(, unbounded queries and JSON. calls on request bodies.

#!/usr/bin/env bash
set -u
SRC="${1:-src}"   # directory to scan, e.g. ./scan-blockers.sh services/api

scan() {
  # $1 = label, $2 = extended regex
  echo "== $1"
  grep -rnE --include='*.js' --include='*.ts' --include='*.mjs' \
       --exclude-dir=node_modules --exclude-dir=dist "$2" "$SRC" || echo "(none)"
  echo
}

scan "Sync calls" '[A-Za-z]+Sync\('

scan "JSON parse/stringify" 'JSON\.(parse|stringify)\('

scan "Nested regex quantifiers" '\([^)]*[+*]\)[+*]'

scan "Array sorts" '\.sort\('

scan "Compile calls" 'compile\('

echo "== SELECT without LIMIT (same line only)"
grep -rniE --include='*.js' --include='*.ts' --exclude-dir=node_modules \
     'select .* from' "$SRC" | grep -vi 'limit' || echo "(none)"

A grep-based audit script that flags likely event-loop blockers (sync calls, JSON work, nested regex quantifiers, sorts, per-request compilation, and queries without LIMIT) so you can review them before you start profiling.

How do you find the culprit with clinic.js or the V8 profiler?

Lag tells you the loop was blocked, not what blocked it. To find the code responsible, you need a profiler that samples the call stack while the stall is happening.

Classify first with clinic.js Doctor

Run clinic doctor -- node server.js and drive realistic load against it, for example with your load-test tool or a replayed traffic sample. Doctor records event loop delay, CPU, memory and active handles, then suggests a likely category: a synchronous event loop problem, slow I/O, or garbage collection pressure. This step matters because the fixes differ. Profiling CPU frames will not help if the real issue is GC pauses from a leak.

Locate hot frames with clinic.js Flame

If Doctor points to blocking work, run clinic flame under the same load. It produces a flame graph from sampled stacks. Width shows how much CPU time a function and its callees used. The horizontal order does not represent time. The pattern to look for is a wide block near the top of a stack: a single synchronous function with high self time, such as JSON.parse, a regex, or a hashing call, sitting directly under your route handler.

No extra dependencies

Node's built-in --cpu-prof flag writes a .cpuprofile file when the process exits. --inspect lets you attach Chrome DevTools and record a profile on demand. Both read the same V8 sampling data.

All of these tools add significant overhead and change timing. Run them in staging or on a single canary instance with reproduced load. Do not run them across the production fleet.

#!/usr/bin/env bash
set -euo pipefail
run_load() { echo "drive realistic load at  here"; }

clinic doctor -- node server.js &
sleep 5 && run_load
wait

clinic flame -- node server.js &
sleep 5 && run_load
wait

node --cpu-prof --cpu-prof-dir=./profiles server.js &
PID=$!
sleep 5 && run_load
kill -TERM "$PID"; wait "$PID" || true
ls ./profiles   # open the .cpuprofile in Chrome DevTools > Performance

node --inspect=127.0.0.1:9229 server.js

A staging profiling workflow that classifies the stall with clinic.js Doctor, locates hot synchronous frames with clinic.js Flame, and falls back to Node's built-in --cpu-prof and --inspect when no extra dependencies are wanted.

What are the alternatives for fixing blocked requests, and what do they cost?

Once profiling points to the blocking function, choose the cheapest fix that takes the work off the main thread. The table lists fixes in order of preference.

Fix Use when Avoid when Dev cost
Avoid the work (pagination, streaming, caching) Payloads or reads grow with data size Clients need the full result at once Low to medium, because the API contract changes
Async fs/crypto/zlib variants A sync call sits in a request path One-time startup code Low
Chunk loops with setImmediate Work splits cleanly and is moderate in size Total job latency matters Medium: the whole job finishes later, and state shared between chunks is harder to reason about
worker_threads or a separate service CPU-bound work such as parsing, image processing or hashing Tasks cheaper than the messaging overhead High
Bound input sizes and check regex safety Any endpoint that accepts user input Never skip this Low

The async variants of fs, crypto and zlib run on libuv's shared threadpool, which is small by default. Under load, the bottleneck can move to threadpool saturation. Tasks then queue and response latency climbs while event loop lag stays flat, so watch request latency as well as lag.

Worker threads have three costs:

  • Spawning a worker takes time, so keep a pool of long-lived workers instead of starting one per request.
  • Messages are copied unless you transfer an ArrayBuffer or use a SharedArrayBuffer.
  • The pool needs its own queue limit, so it rejects excess work instead of buffering it without bound.

As a safety net, shed load when lag crosses a threshold. Returning 503 immediately turns overload into fast failures instead of requests that would time out anyway. Add hysteresis: use separate trip and recovery thresholds, each held for several consecutive samples. This keeps a GC pause or brief spike from rejecting healthy traffic. Tune the thresholds for each service.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to node.js event loop lag, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a route's URL or method automatically notifies the services that consume it — no hunting through repos for callers.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Consumers learn about a breaking change at review time rather than from production errors.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.