What does Nginx do with backend connections by default?

Without any upstream tuning, Nginx talks to backends using HTTP/1.0 and sends a Connection: close header on every proxied request. Each client request gets its own backend TCP connection: a three-way handshake, one request and response, then teardown. If the upstream uses HTTPS, every request also pays for a full TLS handshake, which adds round trips and CPU on both ends. This happens even when the client side is using keepalive, because client connections and upstream connections are managed separately.

The cost shows up as TIME_WAIT sockets on the proxy. Running ss -tan state time-wait on a busy Nginx host often lists thousands of entries pointing at the same backend address. TCP puts the side that sends the first FIN into TIME_WAIT, and with this default behavior Nginx usually closes first. Linux holds each such socket for 60 seconds.

This matters because every connection to one backend IP:port from one proxy IP has to use a different local ephemeral port. Linux's default ip_local_port_range is 32768–60999, which is about 28,000 ports.

Illustrative estimate: 28,000 ports ÷ 60 seconds ≈ 470 new connections per second sustained to a single backend before ports run out. Once they do, connect() fails and Nginx returns errors.

Upstream keepalive fixes this. It gives each worker process a cache of idle backend connections. After a response finishes, the connection goes back into the cache instead of being closed, and the next request reuses it without a new handshake.

#!/usr/bin/env bash
# Inspect TIME_WAIT buildup on an Nginx proxy that uses default (non-keepalive) upstreams.
# BACKEND is a placeholder; set it to your upstream's IP and port.
BACKEND_IP=10.0.1.20
BACKEND_PORT=8080

# Summary counters. The TCP line includes a 'timewait' total.
ss -s

# List TIME_WAIT sockets. With a state filter, ss omits the State column,
# so field 4 is the peer address:port.
ss -tan state time-wait | head -n 5

# Group TIME_WAIT sockets by peer to see which backend is causing the churn.
ss -tan state time-wait \
  | awk 'NR>1 {print $4}' \
  | sort | uniq -c | sort -rn | head -n 10

# Count TIME_WAIT sockets toward one backend IP:port only.
ss -tan state time-wait dst "$BACKEND_IP" "( dport = :$BACKEND_PORT )" \
  | awk 'NR>1' | wc -l

# Show the ephemeral port range available for outbound connections.
# Each connection from one proxy IP to one backend IP:port needs a distinct port
# from this range, including sockets still sitting in TIME_WAIT.
sysctl net.ipv4.ip_local_port_range

Shell commands that use ss to count TIME_WAIT sockets on the proxy, group them by backend address, and compare the count for one backend IP:port against the ephemeral port range reported by sysctl.

How do you enable nginx upstream keepalive correctly?

That cache only works when three settings are all present, and nginx gives no warning when one is missing. It just keeps opening a new connection per request.

keepalive N in the upstream block. This creates the per-worker cache of idle connections. N is the maximum number of idle connections each worker keeps. It does not limit total connections, so a burst can still open more.

proxy_http_version 1.1 in the location. By default nginx talks to backends over HTTP/1.0, where connections close after each response unless both sides negotiate otherwise. HTTP/1.1 makes persistent connections the default.

proxy_set_header Connection "". By default nginx sends Connection: close upstream, which tells the backend to close the connection after the response. Setting the header to an empty string removes it. Note that any proxy_set_header inside a location replaces all headers inherited from the server level, so set this one wherever your other headers are defined.

Directive order

If you use least_conn, hash, or ip_hash, put that directive before keepalive in the upstream block. Otherwise the balancing method may not apply as you intend.

WebSocket upgrades

WebSockets need Connection: upgrade, but regular requests need an empty Connection header. If you hardcode one value in a shared location, you break either upgrades or reuse. Use a map on $http_upgrade instead. When the client sends an Upgrade header, map to upgrade. When it doesn't, map to an empty string. Then pass the mapped variable as Connection and forward Upgrade itself. Many examples map the no-upgrade case to close, which quietly turns off keepalive for normal traffic.

# http context

# Choose the Connection header per request:
# WebSocket handshake -> "upgrade"; everything else -> "" (header removed),
# so normal requests can reuse cached upstream connections.
# Mapping the default to "close" would silently disable keepalive.
map $http_upgrade $connection_upgrade {
    default "upgrade";
    ""      "";
}

upstream app_backend {
    least_conn;                # balancing method goes before keepalive
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;

    keepalive 32;              # max idle connections cached per worker (not a total cap)
}

server {
    listen 80;
    server_name app.example.internal;

    location / {
        proxy_pass

        # Required for reuse: HTTP/1.1 makes upstream connections persistent by default
        proxy_http_version 1.1;

        # proxy_set_header here replaces all server-level ones, so keep them together
        proxy_set_header Host              $host;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Forward the upgrade handshake; empty Connection for regular traffic
        proxy_set_header Upgrade    $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
    }
}

An upstream with least_conn ahead of keepalive, plus a location that uses HTTP/1.1 and a map on $http_upgrade so regular requests reuse connections while WebSocket upgrades still work.

How should you size the keepalive pool?

The number in keepalive N is easy to misread. It sets the maximum number of idle connections each worker process keeps cached for that upstream group. It does not limit how many connections Nginx opens. Under load, a worker opens as many connections as it needs. When requests finish and more than N connections are idle in that worker, it closes the least recently used ones. The total idle capacity is therefore N multiplied by the number of workers, and then by the number of Nginx instances.

Illustrative estimate

Assumed inputs: 3 Nginx instances, 8 workers each, keepalive 32.

Scope Calculation Max idle connections held
Per worker 32 32
Per Nginx instance 8 × 32 256
Whole fleet 3 × 256 768

These connections sit idle and spread across the backend servers in the group. A thread-per-connection backend may allow only 200 concurrent connections, and each idle socket pins a thread. On that backend, 768 cached connections can starve real traffic. The same applies to services with fixed connection slots. Each idle connection also uses a file descriptor and some buffer memory on both ends.

To choose a starting value, find the backend's real connection limit across all its replicas and reserve headroom for health checks and other clients. Then divide the remainder by the total number of Nginx workers. Set N near the steady-state concurrency each worker sees, not the peak. Bursts above N still get served. The extra connections are opened and then closed, and that brief churn is cheaper than holding idle connections the backend cannot spare.

Why do mismatched idle timeouts cause intermittent 502s?

When a backend closes an idle connection, its FIN can still be in flight while Nginx picks that same connection from the pool and writes a request onto it. The backend answers with a reset, and Nginx logs an upstream error that becomes a 502. This only happens when a request lands in the short gap around the backend's idle deadline. It shows up under moderate traffic and almost never in a load test that keeps every connection busy, which is why it is hard to reproduce.

The rule is that Nginx's upstream keepalive_timeout must be shorter than the backend's keep-alive idle timeout. Then Nginx always closes an idle connection before the backend does. The same applies to anything in the path that tracks flows. NAT gateways, L4 load balancers and firewalls can drop idle state without sending anything. In that case Nginx learns the connection is dead only when it writes to it. Set the Nginx timeout below the smallest idle limit along the whole path.

proxy_next_upstream can hide the race. By default, Nginx retries idempotent requests such as GET on another connection, so the client never sees the error. It does not retry non-idempotent requests like POST once they have been sent to the upstream, unless you explicitly allow it. POSTs therefore show the problem, and their error rate is the more honest signal.

Setting Context Purpose
keepalive_timeout upstream Closes idle pooled connections before the backend or any intermediate device does
keepalive_requests upstream Recycles a connection after N requests. This frees memory held per connection and lets long-lived connections rebalance across backends
Backend idle timeout Backend server Must be longer than the Nginx value
Middlebox idle timeout NAT, L4 LB, firewall Must also be longer than the Nginx value

Support for these upstream-level directives and their defaults has changed across Nginx releases. Check the documentation for the build you actually run before relying on them.

# Assumed path limits (placeholders, measure your own):
#   backend keep-alive idle timeout: 75s
#   NAT / L4 load balancer idle timeout: 60s
# Nginx must close idle pooled connections before the smallest of these.

upstream app_backend {
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;

    keepalive 32;              # idle connections cached per worker process
    keepalive_timeout 30s;     # below 60s (middlebox) and 75s (backend)
    keepalive_requests 1000;   # recycle after N requests: frees per-connection
                               # memory and lets traffic rebalance
    # Check that your Nginx build accepts keepalive_timeout and
    # keepalive_requests inside upstream before deploying (nginx -t).
}

server {
    listen 80;

    location / {
        proxy_pass

        # Required for upstream keepalive: HTTP/1.1 and no "Connection: close".
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        # Default behavior: retry on another server after a connection error.
        # Retries hide the race for GET; a POST already sent to the upstream
        # is not retried, so it surfaces as a 502. Adding non_idempotent
        # would retry POSTs too, which risks duplicate writes. Leave it off
        # unless the backend deduplicates requests.
        proxy_next_upstream error timeout;
    }
}

An upstream block whose keepalive_timeout is set below both the backend's and the middlebox's idle timeouts, with keepalive_requests for connection recycling and the default retry policy that masks the race only for idempotent requests.

How does keepalive affect load balancing and HTTPS upstreams?

Nginx runs the balancing algorithm first, then looks in the cache for an idle connection to the server it chose. Within a single upstream block, the pool does not override round-robin or hashing. Two caveats apply, though.

First, least_conn counts only active connections, so idle cached connections are invisible to it. Second, and more important in practice, Nginx can only balance across the addresses it knows about. If the upstream is a single virtual IP in front of an L4 load balancer or a service proxy, Nginx sees one server. Each cached connection stays pinned to whichever backend the L4 layer picked when the connection was opened. After a scale-out, the new instances receive little traffic from Nginx until existing connections are closed by idle timeouts, by hitting keepalive_requests, or by backend restarts. A large pool with a high request limit makes that skew last longer. Lowering the per-connection request limit trades some reuse for faster rebalancing.

For HTTPS upstreams, reuse saves more. A new connection costs a TCP handshake plus a TLS handshake, which adds round trips, asymmetric key exchange and certificate verification. A reused connection skips all of that and sends the request on an already-encrypted channel.

proxy_ssl_session_reuse solves a different problem. It lets Nginx resume a previous TLS session when it has to open a new connection, which gives an abbreviated handshake without the full key exchange. A TCP handshake and at least one TLS round trip still happen. Session reuse makes the remaining new connections cheaper, while keepalive cuts how many new connections you open in the first place.

upstream api_backend {
    # Single VIP in front of an L4 balancer: Nginx sees one server,
    # so each cached connection stays pinned to whatever backend L4 chose.
    server 10.0.0.50:443;

    # Idle connections cached per worker process.
    keepalive 32;

    # Retire each connection after this many requests so the L4 layer
    # gets to re-pick a backend. Lower values rebalance sooner after
    # scale-out but reuse less.
    keepalive_requests 200;

    # Close idle cached connections after this long.
    keepalive_timeout 30s;
}

server {
    listen 80;
    server_name app.example.internal;

    location / {
        proxy_pass

        # Required for upstream keepalive: HTTP/1.1 and no "Connection: close".
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        # Verify the upstream certificate and send SNI.
        proxy_ssl_server_name on;
        proxy_ssl_name api.example.internal;
        proxy_ssl_verify on;
        proxy_ssl_trusted_certificate /etc/nginx/certs/internal-ca.pem;

        # Session resumption only helps when a NEW connection must be opened
        # (abbreviated handshake). Keepalive avoids opening it at all.
        # This is already the default; set explicitly for clarity.
        proxy_ssl_session_reuse on;
    }
}

An HTTPS upstream that reuses cached connections, uses keepalive_requests to recycle them so traffic rebalances behind an L4 VIP, and keeps TLS session resumption on for the new connections it still has to open.

How can you verify that connections are actually reused?

…open. Either way, you should confirm that reuse is actually happening before you rely on it.

Count sockets to the backend port

Run ss -tan on the Nginx host and filter on the backend's destination port. Compare the count of ESTAB sockets with the count of TIME-WAIT sockets, and repeat the check a few times under steady load. When reuse is working, the established count stays roughly flat, near the pool size multiplied by the number of workers, and TIME-WAIT entries for that port stay low. A TIME-WAIT count that rises with request rate means Nginx is still opening and closing a connection for each request.

Log $upstream_connect_time

Add $upstream_connect_time to your access log format. A new connection records the time spent on the TCP handshake, plus the TLS handshake for HTTPS upstreams. A reused connection logs a value at or near zero. If most log lines show zero, the pool is serving most requests.

Check from the backend side

On the backend, count established connections from the Nginx hosts' addresses. The count should be stable rather than rising and falling with each burst of traffic.

Illustrative estimate

These are assumed inputs, not measurements. Take 2,000 requests per second, a 1 ms round trip, and a TLS upstream that needs one TCP round trip and one TLS round trip per new connection. Without reuse, every request pays about 2 ms before any data is sent. Linux holds sockets in TIME_WAIT for 60 seconds, so this rate leaves on the order of 120,000 sockets in that state on whichever side closes first.

Running this check at low traffic confirms the configuration works. If you skip it, port exhaustion can appear later, under peak load, as a 502 error that is hard to diagnose.

When should you avoid upstream keepalive, and what are the alternatives?

Upstream keepalive pays off most when Nginx sends many requests per second to a small set of backends. In that case each new connection repeats the same handshake and leaves another socket in TIME_WAIT on the proxy. The gain is larger with HTTPS upstreams, where reuse also skips the TLS handshake. It is largest on proxies that are already running short of ephemeral ports for a given backend address.

Skip it, or keep the pool small, in three cases:

  • Very low traffic. Idle connections expire before anything reuses them, so you get the maintenance cost with no benefit.
  • Poor handling of persistent connections. Some backends, or intermediate devices such as older load balancers and firewalls, silently drop idle flows or mishandle reused ones.
  • Hard connection limits. If a backend accepts only a few concurrent connections, idle sockets held by each Nginx worker take capacity that other clients need.
Approach Fixes the cause Risk Ongoing cost
Default per-request connections No Port exhaustion and handshake latency under load None
Kernel tuning such as tcp_tw_reuse No Masks churn; depends on TCP timestamps Low
Upstream keepalive Yes 502s if idle timeouts are mismatched Timeout coordination

Kernel tuning only lets the system recycle the extra sockets faster. It does not create fewer of them, so handshake cost and backend accept load stay the same. Some related options are dangerous. tcp_tw_recycle broke clients behind NAT and has been removed from Linux.

The engineering cost of keepalive is a few lines of config, plus an agreement with backend teams that their idle timeout stays longer than the one Nginx uses. That ongoing coordination usually costs less than one incident spent chasing intermittent port exhaustion.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to nginx upstream keepalive, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.