What is the undici connection pool and how does it relate to Node fetch?
Undici is the HTTP/1.1 client behind Node's global fetch. Its design is built around a dispatcher abstraction with three layers. Each layer adds one responsibility on top of the one below it.
| Object | Scope | What it manages |
|---|---|---|
Client |
One origin | A single connection, optionally pipelining several requests over it |
Pool |
One origin | A set of Client instances, handing each request to one with free capacity |
Agent |
Many origins | One Pool per origin, created on demand and keyed by scheme, host and port |
When you call fetch, the request goes to a global dispatcher, which by default is an Agent. The Agent looks up the target's origin and forwards the request to that origin's Pool. The Pool then chooses a Client with a free slot or opens a new connection if it is under its limit. Pool tuning therefore means tuning these per-origin Pool objects, either directly or through the options the Agent passes down when it creates them.
The pooling layer is separate from core Node HTTP. Node bundles its own copy of undici to implement fetch, and that path never touches http.Agent. Setting keepAlive: true or raising maxSockets on http.globalAgent changes nothing for fetch traffic. The setting looks reasonable in code review and still has no effect.
This matters most to backend and platform engineers whose services send a high volume of outbound calls to a few upstreams, such as an internal API, an auth service or a payment gateway. In that pattern, nearly all traffic goes through a handful of pools. Their connection limits and keep-alive behaviour largely determine how much throughput you get.
┌─────────────────────┐ ┌──────────────────────────────┐
│ fetch(url) │ │ http.request / https.request │
└──────────┬──────────┘ └──────────────┬───────────────┘
│ │
▼ ▼
┌─────────────────────────────┐ ┌──────────────────────────────┐
│ global dispatcher: Agent │ │ http.globalAgent (core) │
│ (Node's bundled undici) │ │ keepAlive, maxSockets │
│ one Pool per origin │ │ not used by fetch │
└──────┬────────────────┬─────┘ └──────────────────────────────┘
│ origin A │ origin B
▼ ▼
┌─────────────┐ ┌─────────────┐
│ Pool │ │ Pool │
│ origin A │ │ origin B │
└──┬────────┬─┘ └──────┬──────┘
▼ ▼ ▼
┌──────┐ ┌──────┐ ┌──────┐
│Client│ │Client│ │Client│
└──┬───┘ └──┬───┘ └──┬───┘
│ TCP │ TCP │ TCP
▼ ▼ ▼
┌────────────────┐ ┌──────────┐
│ upstream A │ │upstream B│
└────────────────┘ └──────────┘
fetch sends each request to undici's global Agent, which routes it to a per-origin Pool of Clients that each own one connection, while core http.globalAgent and its keepAlive and maxSockets settings sit on a separate path that fetch never uses.
Who should tune the undici pool, and who can leave it alone?
Pool tuning pays off for services whose outbound HTTP traffic is concentrated on a few origins and is heavy enough to keep connections busy. Typical examples:
- API gateways and backend-for-frontend services that fan each inbound request out to several internal services.
- Workers and batch jobs that call one upstream in a tight loop, such as a queue consumer posting results to an internal API.
- Services in front of a constrained upstream, such as a rate-limited third-party API, a legacy service with a small connection limit, or a proxy that caps connections per client.
- Platform teams that maintain a shared HTTP client wrapper used by many services, where one well-chosen default affects every caller.
You can usually keep the defaults if outbound traffic is light, spread across many origins with only occasional calls to each, or if you see no queueing, connection churn or reset errors. Tuning without a measured symptom adds configuration that someone has to understand and maintain later.
How does an undici pool handle a request from start to finish?
Following one request through the pool explains most of the settings covered in the rest of this article.
- Dispatch.
fetchorrequestpasses the request to a dispatcher. If it is anAgent, theAgentfinds or creates thePoolfor the request's origin. - Client selection. The
Poollooks for aClientthat can accept another request. With the defaultpipelining: 1, that means a client whose connection is open and not currently serving a request. - Growth or queueing. If no client is free and the pool has fewer clients than its
connectionslimit, it creates a newClient, which opens a TCP connection, plus a TLS handshake for HTTPS. If the pool is at its limit, the request waits in the pool's queue. With no limit set, the pool keeps creating clients instead. - Request and response. The client writes the request and parses the response. The caller receives the status and headers and then reads the body as a stream.
- Release. The connection becomes available again only after the response body has been fully read or destroyed. At that point the pool hands the connection to the next queued request, if there is one.
- Idle and close. If nothing is queued, the connection stays open and idle for up to the keep-alive timeout, ready for reuse without a new handshake. When the timeout expires, or the server closes the connection, the socket is discarded and the next request that needs it opens a fresh one.
Each tuning knob maps to one step. connections controls step 3. Unread bodies block step 5. The keep-alive options govern step 6. Timeouts bound steps 3 and 4.
How do you give fetch a configured dispatcher?
Undici gives you two places to attach a dispatcher. The first is per request: Node's fetch accepts a dispatcher option, so the call itself carries the Agent or Pool it should use. The second is process-wide: setGlobalDispatcher replaces the default that every fetch call without an explicit dispatcher falls back to. A global dispatcher is simple, and it also covers fetch calls made inside third-party libraries you don't control. It applies one policy everywhere, though. An Agent keeps a separate pool for each origin, but every origin gets the same connection limit and the same keep-alive timeout. That is a poor fit when your payment gateway and your metrics endpoint behave very differently.
One dispatcher per upstream
Creating one dispatcher per upstream lets you size each pool to match its backend. The cost is lifecycle management. Each dispatcher holds open sockets and timers, so you have to create it once, share it across the codebase, and close it on shutdown. close() lets in-flight requests finish before it releases sockets. destroy() aborts them. If you skip this step, idle keep-alive sockets can keep the event loop alive and slow down a graceful shutdown.
npm undici vs. the bundled copy
Node bundles its own copy of undici, and the undici package you install from npm may be a different release. Defaults, option names and behaviour can differ between the two. Check the documentation for the version you actually import rather than assuming they match.
Confirm the settings took effect
Before you tune anything, check that your dispatcher is actually handling the traffic. Run a load test and count the established TCP connections from the process to the upstream using ss or lsof. If you set connections: 10 and see 40 sockets, or the count never changes when you change the setting, your requests are going through some other dispatcher.
// Import fetch and dispatchers from the same npm undici release so options
// and defaults match the docs for that version, not Node's bundled copy.
import { fetch, Agent, Pool, setGlobalDispatcher } from 'undici';
// Process-wide fallback: one policy for every origin without its own pool.
const globalAgent = new Agent({ connections: 20, keepAliveTimeout: 10_000 });
setGlobalDispatcher(globalAgent);
// One dispatcher per upstream, sized for that backend. Create once, share.
export const payments = new Pool('', {
connections: 10, // strict cap for a slow, rate-limited gateway
keepAliveTimeout: 4_000,
});
export const metrics = new Pool('', {
connections: 50,
keepAliveTimeout: 30_000,
});
// Per request: the call carries the pool it should use.
export async function charge(body) {
const res = await fetch('', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(body),
dispatcher: payments,
});
return res.json();
}
// Lifecycle: close() lets in-flight requests finish, then frees sockets.
// Without this, idle keep-alive sockets can delay a graceful exit.
process.once('SIGTERM', async () => {
await Promise.all([payments.close(), metrics.close(), globalAgent.close()]);
process.exit(0);
});
// Verify under load that the cap applies (expect at most 10 sockets):
// ss -tn state established '( dport = :443 )' | grep -c payments-ip
Sets a global Agent as the fallback, routes payment calls through a dedicated Pool via the per-request dispatcher option, closes every pool on SIGTERM, and uses ss to check that the connection cap is actually enforced.
How many connections should an undici pool allow?
The connections option caps how many sockets a pool opens to a single origin. By default there is no cap, so a pool will open a new socket whenever every existing one is busy. Under steady load that is harmless. Under a burst it is not: a spike in traffic turns into a spike in new connections, and each one pays a TCP handshake and, for HTTPS, a TLS handshake before it carries any request. A small upstream, or one behind a proxy with a modest connection limit, can spend its capacity accepting connections instead of serving requests.
With a cap, excess requests wait in the pool's queue instead of becoming new sockets. That queue lives in your process, so you can measure its depth and the time requests spend waiting before dispatch.
Sizing with Little's law
Little's law says the average number of requests in flight equals arrival rate times mean latency. As an illustrative estimate, assume 500 req/s at 40 ms mean latency: 500 × 0.04 = 20 concurrent requests. A starting cap somewhat above 20 leaves headroom. Widen that headroom if latency varies widely, because slow tail requests hold sockets longer than the mean suggests. Treat the number as a hypothesis and validate it under load.
The trade-off
If the cap is too high, the queue moves into the upstream, where you cannot see it. If the cap is too low, requests wait in your own queue, and that delay looks like upstream slowness in client-side timings.
import { Pool } from 'undici';
// Little's law, illustrative estimate only: assumed inputs, not measurements.
const assumedRate = 500; // req/s
const assumedMeanLatency = 0.04; // seconds (40 ms)
const inFlight = assumedRate * assumedMeanLatency; // ~20
// Start somewhat above the estimate; widen if latency variance is high.
const cap = Math.ceil(inFlight * 1.5); // 30, a hypothesis to validate under load
// Without `connections`, the pool opens a new socket whenever all are busy.
const pool = new Pool('', { connections: cap });
// The queue now lives in this process, so it can be observed.
setInterval(() => {
const { connected, running, queued } = pool.stats;
// Sustained queued > 0 with connected === cap means the cap is binding:
// that waiting time is local queueing, not upstream slowness.
console.log({ cap, connected, running, queued });
}, 1000).unref();
export async function getOrder(id) {
const start = performance.now();
const { statusCode, body } = await pool.request({
path: `/orders/${id}`,
method: 'GET',
});
const data = await body.json(); // always consume the body to free the socket
// This timing includes time spent queued in the pool before dispatch.
const elapsedMs = performance.now() - start;
return { statusCode, data, elapsedMs };
}
Sizes an undici Pool's connections cap from a Little's-law illustrative estimate and samples pool.stats so that queueing caused by the cap shows up as a client-side metric.
How do keep-alive settings prevent ECONNRESET and 'other side closed' errors?
An idle pooled socket is a race. The server keeps its own idle timer, and when it fires the server closes the connection. If the client picks that socket for a new request a moment before the FIN arrives, the write goes to a closed peer, and undici reports ECONNRESET or a SocketError with the message "other side closed". The request never reached your handler, but the client cannot be sure of that.
keepAliveTimeout sets how long undici keeps an idle socket available for reuse. Set it lower than the server's idle timeout so that the client always closes first. When the server sends a Keep-Alive: timeout=N header, undici uses that value instead. It subtracts keepAliveTimeoutThreshold as a safety margin for clock and network delay, and it caps the result at keepAliveMaxTimeout. Keep the threshold larger than your worst-case round-trip time. Otherwise the race still happens on slow links.
Even with careful tuning, you will still see some resets, for example when a peer restarts or a middlebox drops the connection. Retrying is safe for idempotent methods such as GET, PUT and DELETE. Retrying a POST can charge a card twice unless the API accepts an idempotency key.
Long-lived sockets also affect load balancing. A pooled connection stays on the backend IP it resolved and connected to. New instances added behind DNS or an L4 balancer receive no traffic until old sockets close. Shorter idle timeouts, or periodically recycling connections, cost more TCP and TLS handshakes but spread load more evenly.
import { Agent, request } from 'undici';
// Assumes the upstream server closes idle sockets after 5s.
const agent = new Agent({
// Used when the server sends no Keep-Alive hint. Keep it below the
// server's idle timeout so the client closes the socket first.
keepAliveTimeout: 4_000,
// Upper bound on a server-provided Keep-Alive: timeout=N. A low cap also
// limits how long a socket stays pinned to one backend IP.
keepAliveMaxTimeout: 30_000,
// Subtracted from the server hint. Keep it above worst-case RTT.
keepAliveTimeoutThreshold: 1_000,
});
const IDEMPOTENT = new Set(['GET', 'HEAD', 'OPTIONS', 'PUT', 'DELETE']);
// ECONNRESET, or undici's SocketError ("other side closed").
const STALE_SOCKET = new Set(['ECONNRESET', 'UND_ERR_SOCKET']);
export async function send(url, opts = {}, maxAttempts = 2) {
const method = (opts.method ?? 'GET').toUpperCase();
for (let attempt = 1; ; attempt++) {
try {
return await request(url, { ...opts, method, dispatcher: agent });
} catch (err) {
// Never retry POST/PATCH blindly: the first attempt may have been
// processed, so a retry could duplicate side effects.
// Note: a streamed body cannot be resent; use a string or Buffer.
const retryable = IDEMPOTENT.has(method) && STALE_SOCKET.has(err.code);
if (!retryable || attempt >= maxAttempts) throw err;
}
}
}
// Usage (placeholder host):
// const { statusCode, body } = await send('');
// await body.text();
Configures an undici Agent so the client closes idle sockets before the server does, caps how long a socket stays on one backend, and retries stale-socket errors only for idempotent methods.
Why does an undici pool look starved while CPU is idle?
A pool can run out of connections without the process doing any work. In undici, a connection serving a request stays assigned to that request until the response body has been fully read or destroyed. Until then, the socket cannot return to the pool. If your code reads the status and headers but never touches the body, that socket stays occupied. Garbage collection may eventually reclaim it, but it does so late and unpredictably, so you cannot rely on it to free pool slots under load.
With fetch, this most often happens on error paths. A typical leak checks if (!res.ok) throw ... and never reads the 404 or 503 body. Another common case is a request made only for its side effect, where the code ignores the response entirely. Every Response needs one of two endings:
- Consume the body. Use
await res.text()orawait res.arrayBuffer()when the body is small. A fully drained body lets the connection return to the pool and be reused. - Cancel the body. Use
await res.body?.cancel()when the body is large or unbounded. This frees the pool slot quickly, but it may close the connection instead of keeping it alive.
Put this in a finally block so that thrown errors cannot skip it.
The symptom looks like this: requests wait in the pool's pending queue, latency climbs, and caller-side timeouts fire, while CPU and the event loop stay mostly idle. The process is waiting on sockets that nothing is reading.
Raising connections until the errors stop only delays the problem. The leak stays, and you add more sockets on the upstream side. Auditing every fetch call site for unread bodies costs an afternoon, and it removes the cause instead of hiding it.
When should you avoid pipelining and aggressive timeouts?
Undici sends one request per socket at a time by default (pipelining: 1). Each response must finish before the next request goes out on that connection. Keep this default unless you have measured a reason to change it.
Raising pipelining lets the client write several requests onto one socket before the earlier responses arrive. HTTP/1.1 requires responses in request order, so one slow response delays every response queued behind it. This is head-of-line blocking, and it raises p99 latency even when the median improves. A reset on that socket also leaves the client unsure which queued requests the server processed. Only idempotent requests can be retried safely after that. Many proxies and servers also handle pipelining poorly or not at all. Use it only for idempotent traffic to an upstream you have verified.
Timeout layers
| Layer | Measures | Protects against |
|---|---|---|
| Connect timeout | TCP and TLS setup | Unreachable hosts, dropped SYNs |
headersTimeout |
Time until response headers arrive | A server that accepts the request but stalls before replying |
bodyTimeout |
Idle gap between body chunks | A stream that stops sending |
AbortSignal deadline |
Total wall-clock time of the call | Everything, including slow streams |
A headers timeout stops applying once the status line and headers arrive. A server that sends headers quickly and then trickles the body passes that check. bodyTimeout only fires on silence, so a slow but steady stream passes it too. Only an overall deadline enforced with AbortSignal.timeout() caps the total time.
Set these limits per use case, not globally. A 2-second deadline that suits a metadata lookup will abort a large download or a long-lived event stream partway through. Give those calls their own dispatcher, or their own signal, with limits sized to the payload.
What should you monitor, and what are the alternatives to tuning the pool?
Track four signals per origin and give each one a decision it can drive.
- Open versus idle sockets. Recent undici
Poolobjects expose astatsproperty, but the available fields differ between releases, so check what your version provides. If the pool is pinned at its limit with zero idle sockets, it may need more connections. If many sockets sit idle, the limit is higher than the load requires. - Queue depth. A queue that keeps growing means demand exceeds capacity. Before raising
connections, confirm the upstream can handle more concurrency. - New-connection rate. A high rate under steady load means sockets are being thrown away and re-established. Fix this with keep-alive settings, not the connection count.
- Socket wait versus upstream time. Measure how long a request waits for a socket and compare it with how long the upstream takes to respond. If waiting dominates, tune the pool. If upstream time dominates, more sockets only add pressure, and the upstream is what needs fixing.
Alternatives
| Option | Fits when | Development cost |
|---|---|---|
| Keep defaults | Low or bursty traffic, no queueing | None |
undici request API |
Hot paths where fetch's spec overhead matters | Different response API; pin the npm package, which then upgrades separately from Node's bundled copy |
Core http with http.Agent |
Existing code or libraries already built on it | Familiar API, but slower and has its own keep-alive defaults |
| Scale or fix upstream | Upstream latency dominates | Work lands on another team or another service |
Pinning undici from npm gives you control over its version, but you have to track its security releases yourself. Using the copy bundled with Node removes that dependency, but its version changes whenever you upgrade Node.
How can pool tuning save engineering time?
Most of the time saved comes from shorter investigations. Pool problems tend to show up as vague symptoms, such as intermittent resets, rising latency or timeouts with idle CPU, and those symptoms often get blamed on the upstream or the network first. Knowing how the pool works turns them into specific checks.
- Fewer misdirected fixes. Once the team knows that
fetchignoreshttp.globalAgent, nobody spends time adjustingmaxSocketsorkeepAliveon a path that never runs. Checking socket counts withssorlsofconfirms which dispatcher is in use within minutes. - Faster diagnosis of resets. Intermittent
ECONNRESETand "other side closed" errors usually point to a known cause: a client keep-alive timeout that is longer than the server's. Comparing the two timeouts is a quick check, and it avoids a long round of packet captures and cross-team escalation. - Starvation found by audit, not by scaling. When queueing appears with idle CPU, searching call sites for unread response bodies is a bounded task. Raising
connectionsor adding instances only hides the leak, and it comes back as traffic grows. - Metrics that settle ownership questions. Comparing socket wait time with upstream response time shows whether a slowdown belongs to your service or to the upstream. That replaces back-and-forth between teams with a measurement both sides can read.
- Configuration in one place. Defining one shared dispatcher per upstream, with its limits, timeouts and shutdown handling, means individual call sites need no tuning. New code inherits the policy, and reviewers check one module instead of every
fetchcall. - Safer retries. A single retry helper that only retries idempotent methods on stale-socket errors removes the need for each team to write its own, and it avoids incidents caused by duplicated POST requests.
The up-front cost is small: one module that defines dispatchers, a lint rule or review habit for consuming response bodies, and a few per-origin metrics. The return is that the next connection-related incident starts with data instead of guesswork.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to undici connection pool, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.