What is nginx limit_req and how does its leaky bucket work?

limit_req is the directive from Nginx's ngx_http_limit_req_module that throttles requests per key, typically per client IP or API token. It implements a leaky bucket. Each key gets a bucket in a shared memory zone. The configured rate is how fast that bucket drains, and the burst setting is how much it can hold before Nginx starts rejecting requests.

Rates are enforced per millisecond, not per second

The most common surprise is granularity. Nginx tracks rates at millisecond resolution, so 10r/s does not mean "any 10 requests within a given second." It means one request every 100 ms. If a client sends two requests 20 ms apart, the second one exceeds the rate even though the client is nowhere near 10 requests that second.

This is why bursty, low-average clients get rejected when no burst is configured. Consider a single-page app that fires five parallel API calls on page load and then goes quiet for a minute. Its average rate is tiny, but under a bare 10r/s limit, four of those five calls arrive too soon after the first and are refused. Burst exists to absorb that pattern, and later sections cover how to size it.

Why teams use it, and who owns it

Rejecting a request at the proxy costs a shared-memory lookup and a short response. Letting it through can cost a database query, a worker thread, or a downstream call. limit_req protects upstreams cheaply at the edge, before expensive work starts. It belongs to the backend and platform engineers who run Nginx in front of APIs, because tuning it requires knowing both real client traffic patterns and what the upstream can actually sustain.

 Clients (key=IP/token)       Nginx edge: limit_req
+------------------+       +-------------------------------------+
| SPA: 5 calls     |       | shared memory zone, one bucket/key  |
| ~20 ms apart,    |------>|                                     |
| then idle 60 s   |       |   requests in                       |
+------------------+       |        |                            |
                           |        v                            |
                           |   +---------+   bucket full         |
                           |   | bucket  |-----------------+     |
                           |   | holds   |                 |     |
                           |   | burst=N |                 |     |
                           |   +---------+                 |     |
                           |        |  drains at rate:     |     |
                           |        |  10r/s = 1 per 100ms |     |
                           +--------|----------------------|-----+
                                    v                      v
                         +-----------------------+ +---------------------+
                         | Upstream API / DB     | | Rejected at edge    |
                         | expensive work starts | | (503 by default)    |
                         +-----------------------+ +---------------------+

 No burst, 10r/s:  t=0ms -> pass   t=20,40,60,80ms -> rejected

limit_req keeps one leaky bucket per key in a shared memory zone. The bucket drains at the configured rate, enforced per millisecond, and burst sets how much it can hold, so with no burst a short spike of calls is rejected at the edge before it reaches the upstream.

How do you define a limit_req_zone for an API?

A rate limit has two parts. limit_req_zone declares the shared state in the http context, and limit_req applies that state inside a server or location. A minimal setup has three pieces. The first is a zone such as limit_req_zone $binary_remote_addr zone=api_per_ip:10m rate=10r/s;. The second is limit_req zone=api_per_ip; inside location /api/. The third is a proxy_pass to your upstream.

The first argument is the key, meaning whatever identifies a client for counting. Prefer $binary_remote_addr over $remote_addr. The binary form takes a fixed 4 bytes for IPv4 and 16 for IPv6, while the text form is longer and varies in length, so each state entry uses more memory.

The zone=name:size parameter creates a named shared memory area. All worker processes of a single Nginx instance share it. Separate instances do not share it, so with four nodes behind a load balancer a client can reach roughly four times the configured rate.

rate is set in requests per second (r/s) or per minute (r/m). Use r/m for limits below one request per second, such as 30r/m.

When the zone fills, Nginx evicts the least recently used entries. If eviction still cannot free room for a new key, the request is rejected with an error. Size the zone for the peak number of distinct keys you expect, not for average traffic. A scan or botnet creates many keys at once.

As an illustrative estimate, assume about 128 bytes per state. A 10 MB zone then holds roughly 80,000 keys.

http {
    # Key: client IP in binary form (4 bytes IPv4, 16 bytes IPv6) for compact state.
    # Zone: 10 MB shared by all workers of THIS instance only, not across nodes.
    # Rate: 10 requests per second per client IP.
    limit_req_zone $binary_remote_addr zone=api_per_ip:10m rate=10r/s;

    # Sub-1 r/s limits must be written per minute, e.g. one request every 2 seconds.
    limit_req_zone $binary_remote_addr zone=login_per_ip:1m rate=30r/m;

    upstream api_backend {
        server 10.0.0.10:8080;
    }

    server {
        listen 80;
        server_name api.example.com;

        location /api/ {
            # Apply the per-IP limit. Excess requests are rejected (503 by default).
            limit_req zone=api_per_ip;
            proxy_pass
        }

        location = /api/login {
            # Stricter limit for a sensitive endpoint.
            limit_req zone=login_per_ip;
            proxy_pass
        }
    }
}

A minimal configuration that declares per-IP limit_req_zone zones in the http context and applies them with limit_req in API locations that proxy to an upstream, including an r/m rate for a limit below one request per second.

How do burst, nodelay and delay change behavior during a spike?

As an illustrative estimate, take a zone with rate=10r/s, which nginx enforces as one request per 100 ms, and send 25 requests from one key at the same instant. The burst and nodelay settings decide what happens to the 24 requests beyond the first.

Setting Served at once Queued Rejected Worst added latency
no burst 1 0 24 none
burst=10 1 10 14 about 1 s
burst=10 nodelay 11 0 14 none
burst=10 delay=5 about 6 about 5 14 about 0.5 s

Without burst, nginx rejects anything that arrives sooner than 100 ms after the previous request. Real clients rarely space their calls evenly, so a page that fires a few parallel API calls gets errors even when its average rate is well under the limit.

With burst=10, nginx queues the excess and releases one request every 100 ms. Every request is eventually served at the configured rate, but the last one waits about a second. While a request waits, its client connection stays open. If that wait exceeds the client's timeout, the client gives up and may retry, which adds more load to a queue that is already full.

With nodelay, nginx serves the queued requests immediately. Their burst slots still free up at only one per 100 ms. The burst size therefore becomes the number of requests a single key can send to your upstream at the same moment. Size it against what the backend can actually handle, not against what clients would like to send.

delay=N is a middle ground. Requests up to the threshold pass immediately, and the rest of the burst is paced at the configured rate. This parameter is not available in older nginx builds, so confirm that yours accepts it with nginx -t before relying on it.

# http context: one shared state per client IP, 10 requests/second (one per 100 ms)
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;

# Return 429 instead of the default 503 so clients can tell throttling from outages
limit_req_status 429;

upstream api_backend {
    server 127.0.0.1:8080;
}

server {
    listen 80;
    server_name api.example.com;

    # All four locations share zone "api", so one client's requests count
    # against the same budget everywhere. Pick one style per route in practice.

    # No burst: anything sooner than 100 ms after the previous request is rejected
    location /v1/strict/ {
        limit_req zone=api;
        proxy_pass
    }

    # burst=10: up to 10 excess requests wait in a queue, released one per 100 ms.
    # The last one waits about 1 s with its connection held open.
    location /v1/queued/ {
        limit_req zone=api burst=10;
        proxy_pass
    }

    # nodelay: the burst is forwarded immediately, so one client can have
    # up to 11 requests in flight upstream. Size burst to backend capacity.
    location /v1/fast/ {
        limit_req zone=api burst=10 nodelay;
        proxy_pass
    }

    # delay=5: the first excess requests pass at once, the rest of the burst is paced.
    # Not accepted by older builds; run nginx -t to confirm before deploying.
    location /v1/hybrid/ {
        limit_req zone=api burst=10 delay=5;
        proxy_pass
    }
}

One 10r/s zone applied to four API routes. The routes show how no burst, burst, burst with nodelay, and burst with delay handle the same spike from one client.

How do you return 429 responses instead of 503?

By default, Nginx rejects excess requests with 503 Service Unavailable. For an API that is the wrong signal. A 503 tells clients and upstream load balancers that the server is unhealthy, which can trigger failover, alerting, or generic retry logic that makes the spike worse. 429 Too Many Requests tells the client that the server is healthy and that this caller has exceeded its allowance. Set limit_req_status 429; in the http, server, or location context to change the status code.

Rejections are logged at error level by default. During a sustained spike, that can flood the error log and hide real failures. Use limit_req_log_level warn; (or notice/info) to move rejections to a lower level. Nginx logs delayed requests one level below rejected ones, so lowering the level also quiets burst pacing messages.

The default 429 body is an HTML page, which API clients cannot parse. Point error_page 429 at a named location that sets default_type application/json and uses return 429 with a small JSON body containing a stable error code. Clients can then branch on that code instead of string-matching HTML.

Nginx does not add a Retry-After header on its own. You can add one with add_header ... always. The always flag is required because add_header skips error responses without it. The value you set is a fixed number you chose. Nginx does not compute it from the actual state of the bucket for that key, so clients should treat it as a hint and still back off with jitter.

http {
    # 10 MB of shared memory keyed by client IP, refilling at 10 requests/second.
    limit_req_zone $binary_remote_addr zone=api_per_ip:10m rate=10r/s;

    # Return 429 instead of the default 503 so clients and load balancers
    # do not treat rate limiting as an unhealthy server.
    limit_req_status 429;

    # Log rejections at warn instead of error. Delayed requests are logged
    # one level lower (notice), so burst pacing is quieter too.
    limit_req_log_level warn;

    server {
        listen 80;
        server_name api.example.com;

        location /api/ {
            limit_req zone=api_per_ip burst=20 nodelay;

            # Send rejected requests to a JSON handler instead of the HTML page.
            error_page 429 @rate_limited;

            proxy_pass
        }

        location @rate_limited {
            default_type application/json;

            # Static hint chosen by us. Nginx does not compute it from the
            # bucket state. 'always' is required because add_header skips
            # error responses without it.
            add_header Retry-After 1 always;

            # Stable error code that clients can branch on.
            return 429 '{"error":"rate_limited","message":"Too many requests, retry later"}\n';
        }
    }
}

This configuration rate-limits an API by client IP and returns 429 instead of 503, logs rejections at warn level, and sends a JSON error body with a fixed Retry-After hint.

Which key should you rate limit on, and how do you layer limits?

The key determines whose traffic shares a bucket, so choose it before tuning rates.

Key Works well for Weakness
$binary_remote_addr Anonymous, unauthenticated traffic Users behind NAT or corporate proxies share a bucket. Distributed abuse spreads across many IPs.
API key or auth header variable Per-tenant limits on authenticated APIs An unauthenticated client can rotate fake keys, so pair it with an IP limit.
map output, empty for trusted clients Exempting internal services or monitoring Mistakes in the map silently exempt traffic.

Nginx does not count requests whose key evaluates to an empty string. That makes a map the standard way to exempt clients.

You can put several limit_req directives in one context, and each request must pass all of them. A common layout combines a per-key limit with a global or per-endpoint zone. Give login and search their own stricter zones. limit_req counts requests, not cost: a report export and a health check each use one token. Put cheap and expensive endpoints in separate locations with separate rates.

Inheritance gotcha: a location inherits limit_req only when it defines none of its own. If you add one limit_req inside a location, every server-level limit for that location is dropped. You must repeat those limits explicitly.

Behind a load balancer, configure the realip module before you key on the client address. Use set_real_ip_from for each trusted proxy range, real_ip_header X-Forwarded-For, and real_ip_recursive on. Each side of the trust boundary can fail:

  • Trusting too broadly: clients forge the header and pick their own bucket.
  • Trusting nothing: every request carries the balancer's address, so all clients fall into one shared bucket.

How should you test and roll out nginx rate limiting safely?

Rate limiting fails in two ways: limits too tight throttle legitimate clients, and limits too loose protect nothing. Tune from evidence, not guesses.

Generate controlled load first

Before touching production, send traffic at a known rate from a single client. A shell loop of curl calls that prints only the status code is enough to confirm the shape of the limit: the first requests succeed, the burst allowance absorbs the next few, and the rest return 429. For sustained or concurrent load, use a standard generator such as ab or wrk. Since you control the input rate, you can check that nginx enforces the configured rate and burst exactly.

Read both logs

The error log records a "limiting requests" line for each rejection, with the zone name and the excess count. It tells you which zone fired and how far over the limit the client was. The access log tells you how often it happened. Count 429s per client key and per endpoint, and you can tell whether one abusive client is being limited or a whole population is.

Start generous, then tighten

Set initial limits well above anything you expect. Measure real per-client request rates from the access logs over a representative period, including peaks, and lower the limits toward the high end of normal behavior. If your nginx version supports limit_req_dry_run, enable it first. It logs which requests would be rejected but lets them through, so you can validate a limit against live traffic with no client impact.

Why this is cheap to iterate

Each adjustment is a config change: validate it with nginx -t and reload. There is no application build, no deploy pipeline and no coordination with service owners. Feedback arrives in the logs within minutes, so several tuning rounds fit in a single afternoon.

When should you avoid limit_req, and what are the alternatives?

That low cost comes with limits. A limit_req_zone lives in shared memory on a single Nginx instance. Workers on that host share it, but separate hosts do not. Behind a load balancer with N instances, a client spread across them can get close to N times the configured rate. Autoscaling makes this worse: the effective limit rises as you add instances.

limit_req also counts requests, not cost. A cached GET /status and a ten-second report export each use one token. It has no knowledge of plans, tenants or billing tiers unless you encode them in the key. Even then, it cannot express rules such as "10,000 calls per month." It also does not provide backpressure. If a slow database is the real bottleneck, a fixed request rate at the edge will not detect that saturation.

Approach What it limits Scope Adoption cost
limit_req Request rate per key Per instance Config change only
limit_conn Concurrent connections per key Per instance Config change only
Shared-store or gateway limiter Rate or quota per key Global across instances Extra infrastructure, a network hop per check, a failure mode to design for
Application-level quotas Cost, plan, tenant, business rules Global, auth-aware Code, storage, tests, ongoing ownership

Edge limiting alone is enough when its job is to protect capacity from abusive or runaway clients and roughly accurate limits are acceptable. Add a global or application-level layer when limits are contractual, billed, or tied to request cost. Keep limit_req in front of that layer as a cheap first filter.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to nginx rate limiting limit_req, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.