What is nginx upstream keepalive and why does connection churn hurt?

Upstream keepalive is a cache of idle connections that each Nginx worker process keeps open to backend servers. Without the cache, a worker opens a new connection for each proxied request and closes it once the response is done. With the cache, a finished connection goes back into a per-worker pool, and the next request to the same upstream reuses it. The keepalive directive sets the maximum number of idle connections each worker keeps. It is not a cap on total connections.

What churn costs

Each new connection needs a TCP three-way handshake before any request bytes can be sent, which adds at least one round trip of latency. HTTPS backends also need a TLS handshake. That adds more round trips plus asymmetric crypto work on both the proxy and the backend, and at high request rates this CPU cost becomes noticeable.

Closing connections has its own cost. The side that closes first keeps the socket in TIME_WAIT for a period set by the kernel. When that side is the proxy, every closed socket holds a local ephemeral port for the same backend address and port. With enough sustained traffic to one backend, new connect() calls start failing. Illustrative estimate: with about 28,000 usable ephemeral ports and a 60-second TIME_WAIT, a single backend address supports only around 470 new connections per second before ports run out.

Who benefits most

You gain the most from upstream keepalive when you send high request rates to a small set of backends, use HTTPS upstreams, or serve latency-sensitive APIs where a single extra round trip matters. It is worth adding once you proxy steady traffic rather than occasional requests.

Why doesn't adding 'keepalive' to the upstream block work on its own?

The keepalive directive only tells each worker to keep a cache of idle upstream connections. Whether a connection ever reaches that cache depends on the request Nginx sends to the backend. Out of the box, Nginx proxies with HTTP/1.0 and sets Connection: close on every upstream request. The backend does what it is told: it sends the response and closes the socket. The cache stays empty, and every request opens a new TCP connection, plus a TLS handshake if the upstream uses HTTPS. The upstream block looks correct, but nothing changes.

The minimum working setup has three parts:

  • keepalive N in the upstream block, which sets the number of idle connections each worker may hold.
  • proxy_http_version 1.1 in the location, because persistent connections are the default in HTTP/1.1.
  • proxy_set_header Connection "" in the same location. This clears the header so Nginx stops asking the backend to close.

Locations that also proxy WebSockets need extra handling. The upgrade handshake requires Connection: upgrade, so a hardcoded empty value breaks it. Use a map on $http_upgrade instead: when the client sends an Upgrade header, map to upgrade, and when it doesn't, map to an empty string. Then set Connection from the mapped variable. Map the no-upgrade case to an empty string rather than close, which many copied examples use. With close, ordinary API traffic in that location loses connection reuse again.

Directive defaults have changed across Nginx releases. Check the documentation for the version you actually run before you copy this config.

# Choose the Connection header per request (http context).
# Upgrade requests get "upgrade"; everything else gets ""
# (not "close"), so normal traffic can reuse upstream connections.
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      '';
}

upstream app_backend {
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;

    # Idle connections each worker keeps open to this upstream.
    keepalive 32;
}

server {
    listen 80;
    server_name app.example.internal;

    location / {
        proxy_pass

        # HTTP/1.1 makes connections persistent by default.
        proxy_http_version 1.1;

        # Forward the client's Upgrade header for WebSocket handshakes.
        proxy_set_header Upgrade $http_upgrade;

        # "upgrade" for WebSockets, empty otherwise. An empty value
        # stops Nginx from sending its default "Connection: close".
        # For a location with no WebSockets, use: Connection "".
        proxy_set_header Connection $connection_upgrade;

        proxy_set_header Host $host;
    }
}

A keepalive upstream with HTTP/1.1 proxying and a map-based Connection header that serves WebSocket upgrades while keeping upstream connection reuse for ordinary requests.

How does nginx upstream keepalive work inside worker processes?

The number you give keepalive is the maximum number of idle connections each worker process keeps cached. It is not a total cap and does not limit concurrency. When a burst needs more connections, the worker opens them. When they go idle and the cache is already full, the worker closes the least recently used ones. If N is too small for your traffic, you get churn again, even though the directive is set.

                 ┌─ worker 1 ── idle pool (≤ N) ─┐
client ─► nginx ─┼─ worker 2 ── idle pool (≤ N) ─┼─► backend A
                 └─ worker 3 ── idle pool (≤ N) ─┘   backend B

Workers don't share pools. A request handled by worker 2 cannot use a connection worker 1 opened. With many workers and light traffic, each idle pool is visited rarely, so connections can time out before they are reused. That produces low reuse rates even with a correct config. Across the whole server, the number of idle connections can reach N × workers, and the backend must be able to hold that many.

Three directives control how long a cached connection lives:

Directive What it bounds Why it matters
keepalive_timeout How long an idle connection may sit in the pool Set it shorter than the backend's own idle timeout
keepalive_requests How many requests one connection may serve Recycles connections periodically
keepalive_time Total lifetime of a connection Limits long-lived connections, for example to rebalance load

When used inside the upstream block, these directives control backend connections, not client connections. Their availability and defaults depend on the version you run. keepalive_time in particular is missing from older releases. Confirm each one in your version's documentation before relying on it.

              ┌──────────────── nginx ────────────────┐
              │  ┌─ worker 1 ──────────────────────┐  │
              │  │ idle pool: ≤ N  (not shared)    ├──┼──┐
              │  └─────────────────────────────────┘  │  │   ┌───────────┐
┌─────────┐   │  ┌─ worker 2 ──────────────────────┐  │  ├──►│ backend A │
│ clients ├──►│  │ idle pool: ≤ N  (not shared)    ├──┼──┤   └───────────┘
└─────────┘   │  └─────────────────────────────────┘  │  │   ┌───────────┐
              │  ┌─ worker 3 ──────────────────────┐  │  ├──►│ backend B │
              │  │ idle pool: ≤ N  (not shared)    ├──┼──┘   └───────────┘
              │  └─────────────────────────────────┘  │
              └───────────────────────────────────────┘

  N caps IDLE connections per worker, not concurrent ones:
    burst      → worker opens extra connections as needed
    idle > N   → least recently used connections are closed
    server-wide idle connections can reach N × workers

  Cached connection lifetime (confirm each in your version's docs):
    keepalive_timeout   idle time allowed in the pool
    keepalive_requests  requests served per connection
    keepalive_time      total connection lifetime (absent in older releases)

Each Nginx worker keeps its own pool of up to N idle upstream connections that no other worker can reuse, so server-wide idle connections can reach N × workers.

How many keepalive connections should you configure?

The keepalive N value is a per-worker cache of idle connections. It does not cap total connections. The number a backend actually sees is roughly workers × N × Nginx instances, and you should compare that figure against the backend's connection limits. Busy connections above N still open normally. They are simply closed instead of cached once the request finishes.

To size N, start from concurrency ≈ request rate × average backend latency.

Illustrative estimate. Assume the following inputs:

  • 1,500 req/s to one upstream
  • 20 ms average latency
  • 3 Nginx instances with 4 workers each

The calculation:

  1. Total concurrency is 1,500 × 0.02 = 30 in-flight requests.
  2. Per instance, that is 10.
  3. Per worker, it is about 2.5.

Traffic is bursty and workers are not perfectly balanced, so a value of N around 8 covers that with headroom. At quiet times the backend should then expect up to 8 × 4 × 3 = 96 idle connections. Double the instance count during an autoscale event and that ceiling doubles with it.

Those idle connections are not free:

  • On both sides, each one holds a file descriptor and kernel socket buffers. Check ulimit -n and worker_connections on Nginx, and the equivalent descriptor limits on the backend.
  • Thread-per-connection or process-per-connection servers are the most exposed, because every idle socket pins a thread or process. Ninety-six parked connections can consume most of a fixed worker pool and starve new work.
  • APIs that hold a database connection per worker or per client connection can pass the pressure further down, to a database whose connection limit is often far smaller than anything Nginx would notice.

Why does upstream keepalive cause intermittent 502 errors?

The failure is a race on idle connections. Nginx keeps a connection in its cache for up to keepalive_timeout (set inside the upstream block). The backend runs its own idle timer. If the backend's timer is shorter than or equal to Nginx's, the backend can close a socket just as Nginx picks it up for a new request. The request hits a dead connection, and the error log shows upstream prematurely closed connection or Connection reset by peer.

The fix is to keep Nginx's idle timeout below the backend's, with a margin of a few seconds, so Nginx always closes idle connections first.

Component Idle timeout setting Required relationship
Nginx upstream keepalive_timeout in the upstream block Lowest value in the chain
Node.js server.keepAliveTimeout Higher than Nginx's
Go net/http http.Server.IdleTimeout Higher than Nginx's
Gunicorn keepalive setting Higher than Nginx's

Check each runtime's defaults in its own documentation. They differ between versions.

proxy_next_upstream partly hides the problem. On a connection error, Nginx can retry the request on a fresh connection or another server. It does not retry non-idempotent methods such as POST and PATCH by default. The backend may already have processed the request before the connection died. You can enable those retries with the non_idempotent flag, but then a payment or order creation can run twice.

This makes misaligned timeouts a silent failure mode. Most requests succeed, retries absorb the rest, and a small fraction of writes fail with 502s that don't reproduce on demand. That is often harder to diagnose than the connection churn you set out to fix.

upstream app_backend {
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;

    # Idle connections cached per worker process.
    keepalive 32;

    # Must stay BELOW the backend's idle timeout. This assumes the app is
    # configured with a 65s keep-alive (Node keepAliveTimeout, Go IdleTimeout,
    # or Gunicorn keepalive). With 60s here, Nginx always closes idle
    # sockets first and never reuses one the backend is about to drop.
    keepalive_timeout 60s;
}

server {
    listen 80;

    location / {
        proxy_pass

        # Required for upstream keepalive: HTTP/1.1 and no 'Connection: close'.
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        # Retry on a fresh connection or another server when a reused
        # connection fails. POST/PATCH are NOT retried unless
        # 'non_idempotent' is added, which is deliberately left out here:
        # the backend may already have executed the write.
        proxy_next_upstream error timeout;

        # Bound retries so a bad pool does not multiply load.
        proxy_next_upstream_tries 2;
    }
}

An Nginx upstream config where keepalive_timeout is set below an assumed 65-second backend idle timeout, and proxy_next_upstream retries only idempotent requests after a connection error.

How can you verify that Nginx is actually reusing connections?

Run the same checks before and after the change. A config that loads cleanly is not proof that connections are being reused.

Socket counts with ss

On the Nginx host, count sockets to the backend port by state. For example, ss -tan state established '( dport = :8080 )' shows established connections, and the same filter with state time-wait shows TIME_WAIT. Without keepalive, established connections stay low while TIME_WAIT climbs with request rate, because every request opens and closes a connection. With keepalive working, established connections settle near a stable number and TIME_WAIT drops sharply.

TIME_WAIT accumulates on whichever side closes first. Check the backend hosts too, not just Nginx.

$upstream_connect_time in the access log

Add $upstream_connect_time to your log format. It records how long Nginx spent establishing the upstream connection. For a reused keepalive connection there is no connect phase, so the value is zero. For a new connection it shows real handshake time. The share of non-zero values gives you a direct reuse ratio per upstream.

Backend-side signals

Backend access logs that include client port, or server metrics for accepted connections, should show the same Nginx source ports serving many requests, and new-connection rates should fall.

HTTPS upstreams

Over TLS, each new connection pays for a full handshake, and $upstream_connect_time includes that cost. Keepalive avoids the handshake entirely on reused connections. For connections that still get re-established, such as after idle timeouts or backend restarts, proxy_ssl_session_reuse lets Nginx resume a previous TLS session and use an abbreviated handshake. It is enabled by default, so confirm nothing in your configuration turns it off.

#!/usr/bin/env bash
# Run on the Nginx host before and after enabling upstream keepalive, then compare.
set -euo pipefail

BACKEND_PORT=8080
ACCESS_LOG=/var/log/nginx/upstream.log
# Assumes a log_format that ends with $upstream_connect_time, e.g.:
#   log_format upstream '$remote_addr "$request" $status '
#                       '$upstream_addr $upstream_connect_time';

# Count sockets to the backend in a given TCP state (tail drops the header line).
count() {
  ss -tan state "$1" "( dport = :${BACKEND_PORT} )" | tail -n +2 | wc -l
}

# Without keepalive: few established, TIME_WAIT grows with request rate.
# With keepalive: established settles near a stable number, TIME_WAIT drops.
echo "established: $(count established)"
echo "time-wait:   $(count time-wait)"

# Reuse ratio from recent requests: a connect time of 0 means the request
# went over an existing keepalive connection. Non-zero means a new TCP
# connection (plus a TLS handshake for HTTPS upstreams).
tail -n 10000 "$ACCESS_LOG" | awk '
  $NF == "-" { next }                  # request never reached an upstream
  { total++; if ($NF + 0 == 0) reused++ }
  END {
    if (total) printf "reused: %d/%d (%.1f%%)\n", reused, total, 100 * reused / total
    else print "no upstream requests found"
  }'

A repeatable before/after check that counts established and TIME_WAIT sockets to the backend with ss and calculates the keepalive reuse ratio from $upstream_connect_time in the access log.

When should you avoid upstream keepalive, and what are the alternatives?

Keepalive is a poor fit when even load distribution matters more than handshake savings. Nginx balances per request, but a pooled connection stays pinned to the backend it first reached. If the upstream is a single address in front of a TCP (L4) load balancer, that balancer only places connections, not requests. A new instance added during a scale-out then receives little traffic until existing connections close. keepalive_requests and keepalive_timeout in the upstream block control this. They cap how many requests a connection serves and how long it sits idle, so lower values force reconnects and give the balancer more chances to rebalance. Backends with strict connection limits are the other risk. Idle pools across every worker on every Nginx node add up, and they can use up a backend's connection budget before it has handled much work.

Approach Gains Costs
No keepalive, tuned TIME_WAIT handling and wider local port range Even distribution, simple model Handshake on every request, kernel tuning on every proxy host
Short keepalive with aggressive recycling Most of the reuse benefit, regular rebalancing More reconnects, tuning needed per traffic level
Backend-side connection pooler or L7 load balancer Request-level balancing, central limits Another hop and another component to run
HTTP/2 or gRPC to upstreams, where your Nginx build supports it Many requests over one connection Depends on module support, and failures affect more traffic

The development cost of keepalive is small: a few directives. The operations cost continues. Every backend timeout change has to be checked against Nginx's idle timeout, and pool sizes need review as worker counts, proxy nodes and backend instances grow.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to nginx upstream keepalive, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.