How does Nginx proxy gRPC traffic over HTTP/2?

Nginx proxies gRPC by terminating the client's HTTP/2 connection and opening its own HTTP/2 connection to the backend. gRPC depends on HTTP/2 features: binary framing, multiplexed streams, and trailers. The final grpc-status and grpc-message arrive as trailing headers after the response body. proxy_pass speaks HTTP/1.x to upstreams and cannot relay that framing or the trailers, so a call routed through it fails even when the client-side connection looks healthy. grpc_pass exists for this reason. It forwards to grpc:// (cleartext HTTP/2) or grpcs:// (HTTP/2 over TLS) upstreams.

On the client side, the listener must speak HTTP/2. In practice that means TLS with ALPN, where client and server agree on h2 during the handshake. Cleartext HTTP/2 with prior knowledge also works for internal clients that are configured for it. The syntax for enabling HTTP/2 on a listener has changed across Nginx releases: older configurations use a listen parameter, and newer ones use a separate directive. Check the documentation for your version.

The request path

  1. Per connection. The client connects, and Nginx selects a server block, completes TLS (using SNI for certificate choice), and negotiates h2 through ALPN.
  2. Per request. Each RPC is a separate HTTP/2 stream: a POST to /package.Service/Method with content-type: application/grpc. Nginx matches that path against location blocks, so you can route by service or by individual method.
  3. Upstream. For each call, Nginx picks a backend from the matched upstream group. It does not pin a client connection to one backend, which is why balancing works per call rather than per connection.
+--------------------------+
| gRPC client              |
| calls Users/Get,         |
|       Orders/List        |
+--------------------------+
        |  ONE connection: TCP + TLS, ALPN agrees on h2
        |  (or cleartext h2 with prior knowledge, internal)
        v
+------------------------------------------------------------+
| Nginx                                                      |
| PER CONNECTION: pick server block, TLS cert via SNI, h2    |
|------------------------------------------------------------|
| PER REQUEST: each RPC = one HTTP/2 stream                  |
|  POST /pkg.Users/Get    content-type: application/grpc     |
|    -> location /pkg.Users/   -> grpc_pass grpc://users     |
|  POST /pkg.Orders/List  content-type: application/grpc     |
|    -> location /pkg.Orders/  -> grpc_pass grpcs://orders   |
+------------------------------------------------------------+
        |  Nginx's own HTTP/2           |  backend chosen
        |  connections to backends      |  per call, not
        v                               v  per client conn
+------------------------+    +------------------------+
| upstream users         |    | upstream orders        |
| grpc://  (cleartext h2)|    | grpcs:// (h2 over TLS) |
|  +------+  +------+    |    |  +------+  +------+    |
|  |  u1  |  |  u2  |    |    |  |  o1  |  |  o2  |    |
|  +------+  +------+    |    |  +------+  +------+    |
+------------------------+    +------------------------+
  replies: DATA frames, then trailers (grpc-status, grpc-message)
  relayed back on the same streams; proxy_pass (HTTP/1.x) cannot.

Nginx terminates the client's HTTP/2 connection, makes TLS/SNI/ALPN decisions once per connection, routes each RPC stream by its /package.Service/Method path to a grpc_pass upstream, and picks a backend per call while relaying trailers back to the client.

Why put Nginx in front of gRPC services, and who should?

The main reason is consolidation. gRPC services each need TLS certificates, client authentication, rate limits and access logging. Without a proxy, every service implements those separately, and they drift. Nginx gives you one place to terminate TLS, enforce mutual TLS or IP allowlists, and apply request limits before traffic reaches application code. Rotating a certificate becomes one reload instead of a redeploy across a fleet.

Routing is the second reason. A gRPC method is an HTTP/2 request to a path shaped like /package.Service/Method, so ordinary location blocks can send one service, or one method, to a different upstream group. You can split a monolith gradually, or isolate an expensive method on its own backends, without changing clients.

The third reason is operational. If your team already deploys, monitors and debugs Nginx, adding gRPC reuses that knowledge. You do not have to adopt a new proxy, control plane or configuration model.

Where it fits

The fit is best for platform teams that already run Nginx, for workloads made mostly of unary or short-lived RPCs, and for a modest number of services whose configuration can be maintained by hand or with templates.

What Nginx does not understand

Nginx treats gRPC largely as HTTP/2 with trailers. It can retry a failed upstream attempt, but it has no per-method retry policy keyed on gRPC status codes. It does not consume gRPC-aware load reports from backends. It logs requests rather than individual messages inside a stream, so per-message visibility has to come from the services.

How do you configure grpc_pass, routing and TLS?

Start with a server block that listens on 443 with ssl and HTTP/2 enabled. Depending on your Nginx release, you enable HTTP/2 either with the http2 parameter on listen or with the standalone http2 on; directive. gRPC clients will not fall back to HTTP/1.1, so a listener without HTTP/2 fails on every call. Point ssl_certificate and ssl_certificate_key at the certificate chain and key that clients will validate.

Define each backend pool in an upstream block, then route by path. Every gRPC method is a POST to /package.Service/Method, so a location /orders.v1.OrderService/ block sends the whole service to one pool. You can version, split or migrate services without the client knowing about the change. Inside each location, grpc_pass grpc://orders_backend; speaks cleartext HTTP/2 (h2c) to the upstream, and grpcs:// opens TLS instead.

To re-encrypt, configure these directives:

  • grpc_ssl_verify on; makes Nginx validate the backend certificate. Without it, the upstream TLS only encrypts the traffic and does not authenticate the backend.
  • grpc_ssl_trusted_certificate points at your internal CA bundle.
  • grpc_ssl_server_name on; sends SNI. Use grpc_ssl_name to set the name that is checked against the certificate.
  • grpc_ssl_certificate and grpc_ssl_certificate_key present a client certificate, which gives you mutual TLS to backends that require it.

A plaintext backend hop is easier to operate and debug. It is reasonable when the network segment is already trusted. Re-encrypting adds handshake and per-byte CPU cost on both sides. It also requires certificate issuance and rotation for every backend as well as for Nginx's client identity. Choose re-encryption when policy or a zero-trust network model requires it.

upstream orders_backend   { server 10.0.1.10:50051; server 10.0.1.11:50051; }
upstream payments_backend { server 10.0.2.10:50052; server 10.0.2.11:50052; }

server {
    listen 443 ssl;
    http2 on;  # older releases: listen 443 ssl http2;
    server_name grpc.example.internal;

    # Chain and key that gRPC clients validate
    ssl_certificate     /etc/nginx/tls/grpc.example.internal.fullchain.pem;
    ssl_certificate_key /etc/nginx/tls/grpc.example.internal.key;

    # Trusted segment: cleartext HTTP/2 (h2c) to the backend
    location /orders.v1.OrderService/ {
        grpc_pass grpc://orders_backend;
    }

    # Re-encrypt with mutual TLS to backends that require it
    location /payments.v1.PaymentService/ {
        grpc_pass grpcs://payments_backend;

        # Authenticate the backend, not just encrypt the hop
        grpc_ssl_verify             on;
        grpc_ssl_trusted_certificate /etc/nginx/tls/internal-ca.pem;
        grpc_ssl_server_name        on;  # send SNI
        grpc_ssl_name               payments.svc.internal;  # name checked against backend cert

        # Nginx's client identity for mTLS
        grpc_ssl_certificate        /etc/nginx/tls/nginx-client.pem;
        grpc_ssl_certificate_key    /etc/nginx/tls/nginx-client.key;
    }
}

An HTTP/2 TLS listener that routes by /package.Service/ prefix, sending one service to a plaintext h2c pool and re-encrypting another with verified mutual TLS.

Which Nginx timeouts break gRPC streaming calls?

A unary call rarely trips Nginx timeouts. Streaming calls do, because Nginx measures inactivity between reads and writes, not total call duration.

Directive What it measures Default Typical failure
grpc_connect_timeout Establishing the upstream connection 60s Slow failover to a dead backend
grpc_read_timeout Gap between two successive reads from upstream 60s Idle server stream cut off
grpc_send_timeout Gap between two successive writes to upstream 60s Stalled client stream aborted
client_body_timeout Gap between reads of the client request body 60s Quiet client-streaming call dropped

Here is the classic failure. A client opens a server-streaming subscription with a 30-minute deadline. The server emits events only when something changes. If 60 seconds pass with no message, Nginx closes the upstream stream. The client sees an error status, not DEADLINE_EXCEEDED, so it looks like a server bug. The client deadline is useless here because Nginx enforces its own, shorter limit.

Raise these timeouts only in the location blocks that match streaming method paths. Leave unary routes at short values. A global ten-minute read timeout lets every hung unary call hold a worker connection and an upstream stream for ten minutes. A better fix is to have the server send periodic application-level heartbeats on the stream.

Connection lifetime matters as well. Without a keepalive directive in the upstream block, Nginx does not reuse upstream connections. On the client side, keepalive_timeout and the per-connection request limit cause Nginx to send GOAWAY. Nginx terminates HTTP/2 PING frames rather than forwarding them, so client keepalive pings never reach the backend. Configure client pings at an interval longer than the backend's idle timeouts, and make sure clients handle GOAWAY by reconnecting.

upstream events_backend {
    server 10.0.0.11:50051;
    server 10.0.0.12:50051;
    keepalive 32;               # reuse idle upstream HTTP/2 connections
    keepalive_timeout 60s;      # close idle upstream connections after this
}

server {
    listen 443 ssl;
    http2 on;                   # nginx 1.25.1+; older: listen 443 ssl http2;
    ssl_certificate     /etc/nginx/tls/grpc.example.internal.crt;
    ssl_certificate_key /etc/nginx/tls/grpc.example.internal.key;

    keepalive_timeout  300s;    # idle client connection limit, then GOAWAY
    keepalive_requests 10000;   # streams per client connection before GOAWAY
    http2_max_concurrent_streams 128;

    # Unary methods: short limits so hung calls release resources quickly.
    location / {
        grpc_connect_timeout 5s;
        grpc_read_timeout    30s;
        grpc_send_timeout    30s;
        grpc_pass grpc://events_backend;
    }

    # Server-streaming subscription: may be quiet between events.
    # Keep this above the server's heartbeat interval, not the client deadline.
    location = /events.v1.EventService/Subscribe {
        grpc_connect_timeout 5s;
        grpc_read_timeout    300s;  # max gap between upstream messages
        grpc_send_timeout    30s;
        grpc_pass grpc://events_backend;
    }

    # Client-streaming upload: the client may pause between messages.
    location = /events.v1.EventService/Publish {
        client_body_timeout  300s;  # max gap between client body reads
        grpc_send_timeout    300s;
        grpc_read_timeout    30s;
        grpc_pass grpc://events_backend;
    }
}

Shows short default gRPC timeouts for unary routes, per-location overrides for the streaming methods only, upstream keepalive for connection reuse, and the client-facing keepalive and HTTP/2 stream limits that control when Nginx sends GOAWAY.

How does gRPC load balancing work in Nginx with long-lived streams?

Nginx balances each RPC independently, not each client connection. A unary call goes to whichever backend the algorithm picks. A streaming RPC stays on its chosen backend until the stream ends. If you add backends during a scale-up, they receive little traffic while existing streams keep running. Load evens out only as streams finish or clients reconnect, so give long-lived streams a server-side maximum lifetime.

Method Picks backend by Fits
Round robin (default) Rotation, optionally weighted Short unary calls of similar cost
least_conn Fewest active connections Mixed or long-lived streams
hash $key consistent Key such as a metadata header Cache or session affinity

Failures and retries

max_fails and fail_timeout are passive checks. Nginx marks a server down only after real requests to it fail. Open-source Nginx does not implement the gRPC health checking protocol, so a backend reporting NOT_SERVING still receives traffic as long as it accepts connections.

grpc_next_upstream retries on another server. Every gRPC call is a POST, which Nginx treats as non-idempotent. Without the non_idempotent flag, Nginx will not retry a request after sending it upstream. Enable that flag only for methods that are safe to run twice, such as reads. Otherwise a retried payment or write can execute twice.

Errors and body size

By default Nginx answers errors with HTML, which gRPC clients cannot parse. Use error_page to send these codes to internal locations that return the matching grpc-status:

  • 502 → 14 UNAVAILABLE
  • 504 → 4 DEADLINE_EXCEEDED
  • 413 → 8 RESOURCE_EXHAUSTED

client_max_body_size defaults to 1 MB. It counts the whole request body, so raise it for large unary payloads and client-streaming routes. Nginx streams gRPC request bodies to the backend instead of buffering them. Once part of a body has been sent, Nginx cannot replay it, so that request cannot be retried.

upstream grpc_backend {
    least_conn;  # long-lived streams make active-connection count the better signal
    server 10.0.0.11:50051 max_fails=3 fail_timeout=10s;  # passive: down only after real failures
    server 10.0.0.12:50051 max_fails=3 fail_timeout=10s;
}

server {
    listen 443 ssl;
    http2 on;
    ssl_certificate     /etc/nginx/tls/server.crt;
    ssl_certificate_key /etc/nginx/tls/server.key;
    client_max_body_size 16m;  # whole request body, including client-streamed messages

    error_page 502 = /grpc_unavailable;
    error_page 504 = /grpc_deadline;
    error_page 413 = /grpc_too_large;

    location / {
        grpc_pass grpc://grpc_backend;
        # Writes: retry only if the request was never sent upstream
        grpc_next_upstream error timeout;
    }

    location /inventory.v1.InventoryService/GetItem {
        grpc_pass grpc://grpc_backend;
        # Read-only, safe to run twice, so allow retry after sending
        grpc_next_upstream error timeout non_idempotent;
        grpc_next_upstream_tries 2;
    }

    # Trailers-only style responses gRPC clients can parse instead of HTML
    location = /grpc_unavailable { internal; default_type application/grpc; add_header grpc-status 14; add_header grpc-message "unavailable"; return 204; }
    location = /grpc_deadline    { internal; default_type application/grpc; add_header grpc-status 4;  add_header grpc-message "deadline exceeded"; return 204; }
    location = /grpc_too_large   { internal; default_type application/grpc; add_header grpc-status 8;  add_header grpc-message "request too large"; return 204; }
}

A gRPC proxy using least_conn with passive max_fails checks, retries that include non_idempotent only on a read-only method, a raised client_max_body_size, and error_page mappings that return grpc-status 14, 4 and 8 instead of HTML for 502, 504 and 413.

How do you verify an nginx grpc proxy is working?

Start with grpcurl against the Nginx listener, not the backend. If the backends expose server reflection, list and describe confirm that TLS negotiation, ALPN selection of HTTP/2 and path routing all work end to end. If you route by service name in separate location blocks, the reflection service needs its own route, or listing fails while real calls succeed. A call to grpc.health.v1.Health/Check is the cleaner smoke test, because it returns a defined SERVING status and needs no application payload.

Then make a call that should fail, such as a method name the backend does not implement. You should get Unimplemented from the backend. If you get Unavailable, Internal or an HTTP-level error instead, Nginx is generating the response itself. That usually points to a protocol mismatch, such as grpc:// against a TLS backend, or to a missing route.

For distribution, define a log_format that records $status, $upstream_addr, $request_time and $upstream_trailer_grpc_status. The trailer variable matters because gRPC failures typically still return HTTP 200, so $status alone hides them. Run a batch of unary calls and check that $upstream_addr rotates across backends. A request with a very long $request_time that is always tied to the same upstream is a stream pinned to one backend. Its duration also tells you which timeout will cut it off.

Running these checks before rollout surfaces truncated streams and protocol errors in a controlled test. Diagnosing the same problems from vague client-side Unavailable errors in production takes much longer.

When should you avoid Nginx for gRPC, and what are the alternatives?

Nginx starts to fight you under four conditions:

  • Heavy bidirectional streaming. Each stream stays on the backend it started on. Long-lived streams therefore skew load, and the workarounds (stream lifetime caps, forced reconnects) push extra logic into clients.
  • Frequent autoscaling. New pods receive no existing streams. Scaled-down pods drop the streams they hold.
  • Per-method retry policies. Nginx retries upstream failures as a whole and does not read gRPC status codes per method.
  • Health-aware routing. Open-source Nginx detects failures passively, by watching failed requests, rather than probing the gRPC health service.
Approach Stream distribution Central TLS/policy Where complexity lives
Nginx Per connection, coarse Yes Proxy config, workarounds
gRPC-native proxy Per request/stream Yes A new component to run
Client-side (DNS or lookaside) Per request, fine-grained No Every client, every language

Client-side balancing spreads streams best because each client opens connections to many backends. The cost is that every service must ship correct resolver and balancer config. You also lose the single place to terminate TLS and enforce policy.

Adoption cost favours Nginx if your team already runs it. Operational cost grows over time from streaming workarounds, certificate rotation for re-encryption to backends, and custom mapping of proxy errors to gRPC status codes.

Decision checklist

  • Mostly unary calls or short streams? Nginx is fine.
  • Long bidirectional streams with autoscaling? Use a gRPC-native proxy.
  • Need retries keyed to status codes per method? Use a gRPC-native proxy.
  • Few client languages and a strong platform team? Consider client-side balancing.
  • Need one TLS and policy choke point? Keep a proxy.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to nginx grpc proxy, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.