How does Nginx limit_req work as a leaky bucket?
limit_req_zone defines the limiter: the key to track (commonly the client address), a named shared memory zone that every worker process reads and writes, and a rate. limit_req applies that zone to a server or location. A minimal pair is limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s; in the http block and limit_req zone=api; inside location /api/.
The rate is not a counter that resets every second. Nginx stores an "excess" value per key, tracked in milliseconds. That value drains continuously at the configured rate, so 10r/s means one request every 100 ms. It does not mean "any 10 requests within a given second." Each new request adds one unit to the excess. If the excess stays within the allowed burst, the request proceeds. A positive excess is delayed until the bucket drains far enough for it to fit. Anything beyond that is rejected with 503 by default.
Because the timing is per-millisecond, two requests arriving 20 ms apart already exceed 10r/s, even if the client sends only two requests that second. The timeline below assumes rate=10r/s with burst=2. A later section covers burst in detail. Without it, every early request is rejected.
| Arrival (ms) | Excess after | Outcome |
|---|---|---|
| 0 | 0 | accepted immediately |
| 10 | 0.9 | queued, served ~100 |
| 20 | 1.8 | queued, served ~200 |
| 30 | 2.7 > 2 | rejected (503) |
| 250 | 0.5 | queued, served ~300 |
| 400 | 0 | accepted immediately |
The rejected request does not add to the excess. That is why the request at 250 ms only waits until the next free 100 ms slot.
http {
# Limiter definition (http context only):
# key = client IP in binary form (4 bytes for IPv4, 16 for IPv6)
# zone = shared memory segment "api", 10 MB, read and written by all workers
# rate = 10r/s, which nginx treats as one request per 100 ms, drained continuously
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
upstream api_backend {
server 127.0.0.1:8080; # placeholder backend
}
server {
listen 80;
server_name api.example.com;
location /api/ {
# Apply the zone. burst=2 lets up to 2 excess requests wait in the
# queue and be released at the 100 ms drain pace instead of being
# rejected outright. Without burst, any request that arrives before
# the previous one has drained is rejected immediately.
limit_req zone=api burst=2;
# Rejected requests get 503 by default. This line only states
# the default explicitly.
limit_req_status 503;
proxy_pass
}
}
}
A minimal limit_req_zone and limit_req setup that drains at one request per 100 ms per client IP, queues up to two excess requests, and rejects the rest with the default 503.
How does it work, step by step, for a single request?
The bucket model above describes the math. This is the sequence Nginx follows for each request that reaches a location with limit_req:
- Evaluate the key. Nginx computes the zone's key expression for the request, for example
$binary_remote_addror a variable built withmap. If the key evaluates to an empty string, that zone does not count the request at all. - Look up the state in shared memory. The key is looked up in the zone's shared memory. If no state exists, Nginx allocates one. If the zone is full, it evicts least recently used entries to make room. If it still cannot allocate, the request is rejected.
- Update the excess. Nginx subtracts the amount drained since the last accepted request (elapsed time multiplied by the rate), then adds one unit for the current request.
- Decide. If the new excess exceeds
burst, the request is rejected with thelimit_req_statuscode and the stored excess is left unchanged. If the excess is withinburst, the request is accepted. It is forwarded immediately when the excess is zero, whennodelayis set, or when the excess is withindelay=N. Otherwise Nginx sets a timer and holds the request until enough of the bucket has drained. - Apply every limit. When several
limit_reqdirectives apply to the same context, each zone is checked. A rejection by any zone rejects the request. If more than one zone wants to delay it, the longest delay applies. - Log and expose the outcome. Rejections and delays are written to the error log at the configured
limit_req_log_level. The result is available in$limit_req_statusfor access logging.
All of this happens before the request is proxied, so rejected and not-yet-released requests never reach the backend. Delayed requests still hold a client connection and a small amount of Nginx memory while they wait.
Why does limit_req without burst reject legitimate API clients?
Rate limiting exists to protect backend capacity. One client in a retry loop, a scraper walking every ID, or a credential-stuffing script can consume worker threads and database connections that other users need. limit_req drops that excess at the edge, before it reaches application code. The problem is that a bare rate applies the same strict spacing to well-behaved clients.
Nginx enforces the rate at millisecond granularity. A zone set to 10r/s does not mean "ten requests in any given second." It means one request every 100 ms. Without burst, the allowed excess is zero, so any request that arrives sooner than 100 ms after the previous accepted one is rejected.
Real clients rarely space their requests evenly. A single-page app loading a dashboard might fetch the user profile, feature flags, notifications, and three widgets in parallel, all within a few milliseconds. With no burst, the first request succeeds and the other five get errors. Average traffic from that user is well under the limit, but the instantaneous arrival pattern is not. Mobile clients that batch requests after reconnecting, and backend services fanning out calls, show the same pattern.
Measure this before you pick values. Log $msec alongside the key you plan to limit on, such as the client address or an API token. Then group the log by key and look at two numbers: how many requests arrive within the same 100–500 ms window during a normal page load, and the sustained per-second rate over longer periods. The first number sets the floor for burst. The second sets rate.
What do burst, nodelay, and delay actually change?
With burst, requests that exceed the rate are queued instead of rejected. Nginx then releases them at the zone's configured rate. With limit_req zone=api burst=20; on a 10r/s zone, Nginx frees one slot every 100 ms. The 20th queued request therefore waits about 2 seconds before it is forwarded. This is an illustrative estimate that assumes 20 excess requests arrive at once. The backend sees a smooth flow, but the client sees added latency. Each queued request also keeps its client connection open while it waits.
Adding nodelay (limit_req zone=api burst=20 nodelay;) forwards queued requests immediately. The bucket still drains at 10r/s, so the burst slots stay occupied and refill at that rate. A client that uses all 20 slots gets errors until the slots drain. nodelay removes the added latency but keeps the cap on volume. The tradeoff is that the backend receives the whole spike at once.
delay=N sets up two-stage limiting. With limit_req zone=api burst=20 delay=8;, the first 8 excess requests go through without delay. The next 12 are queued and released at the rate. Anything beyond 20 is rejected. This suits a page load that fires a handful of parallel calls, which should not be slowed, while still throttling a client that keeps pushing traffic.
| Mode | Latency added | Backend spike size | Connection hold time | Best-fit client |
|---|---|---|---|---|
burst (queued) |
Up to burst ÷ rate | Smoothed to rate | Long, while queued | Batch jobs and tolerant scripts |
burst + nodelay |
None | Full burst at once | Short | Interactive SPAs and mobile apps |
burst + delay=N |
None for the first N, then queued | N at once, then smoothed | Moderate | Mixed traffic, such as browsers plus integrations |
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
upstream api_backend {
server 127.0.0.1:8080; # placeholder backend
}
server {
listen 80;
limit_req_status 429; # reject excess with 429 instead of the default 503
# Mode 1: queue. Up to 20 excess requests wait and are released every 100 ms.
# The last queued request waits about 2 s (illustrative estimate: 20 / 10r/s).
location /batch/ {
limit_req zone=api burst=20;
proxy_pass
}
# Mode 2: no queueing delay. Up to 20 excess requests are forwarded immediately.
# Slots still refill at 10r/s, so a client that uses all 20 is rejected until they drain.
location /app/ {
limit_req zone=api burst=20 nodelay;
proxy_pass
}
# Mode 3: two-stage. The first 8 excess requests pass immediately,
# the next 12 are queued at the rate, and anything beyond 20 is rejected.
location /api/ {
limit_req zone=api burst=20 delay=8;
proxy_pass
}
}
One 10r/s zone applied in the three limit_req modes: queued burst, burst with nodelay, and two-stage limiting with delay=8. Each location sends excess requests through a different path.
When should you use nodelay, delay, or a queued burst?
For interactive, user-facing APIs, burst with nodelay is usually the better default. A browser or mobile client that fires a dozen calls on page load is waiting on all of them. Queuing those calls adds latency the user sees directly, and an unhurried response is often worse than a fast 429 the client can handle. With nodelay, requests inside the burst allowance go straight through, and the bucket still refills at the configured rate. A client that keeps hammering gets rejected rather than slowed. The cost moves to the backend: it must absorb a spike of burst-sized requests from every client at once. If 500 clients each have burst=20, your upstream must handle up to 10,000 near-simultaneous requests without falling over.
A queued burst (burst without nodelay) suits batch jobs, webhooks, sync agents, and other machine clients that tolerate latency and would rather wait than retry. It also suits backends that degrade badly under spikes, such as a database-heavy endpoint or a legacy service with a small worker pool, because Nginx releases requests at a steady rate. The trade-off is that every queued request holds a client connection open in Nginx while it waits. Under sustained abuse, that consumes worker connections and file descriptors, and slow clients can sit in the queue longer than their own timeouts allow.
Use delay=N when clients send a short initial burst followed by steady traffic. A dashboard that loads eight widgets and then polls is a typical example. The first N excess requests pass immediately, which keeps initial loads fast, and anything beyond that is smoothed. Set N to match the observed size of the legitimate initial burst, not the full burst value.
Who should use limit_req with burst, nodelay, and delay?
These directives are aimed at teams that already run Nginx in front of an HTTP API and want a first line of protection without changing application code. Typical users:
- Platform and SRE teams operating a reverse proxy or ingress layer in front of several services. They can apply consistent per-client limits in one place instead of asking each service team to build its own.
- Backend teams with a fragile or expensive upstream, such as a database-heavy endpoint, a legacy service with a small worker pool, or a login handler doing costly password hashing. A queued burst or a strict per-location zone keeps spikes from reaching it.
- API owners serving browsers, mobile apps, and partner integrations from the same endpoints.
delay=Nlets interactive page loads through quickly while smoothing heavier machine traffic. - Small teams without a shared rate-limit store.
limit_reqneeds only Nginx and shared memory on the host, so it works before you have the infrastructure for a centralized limiter.
It fits less well if you need exact, globally consistent quotas across many Nginx nodes, billing-aware tiers, or protection against large distributed floods. The final section covers those cases and the alternatives.
How can it save engineering time?
Most of the savings come from handling abusive or buggy traffic at the edge, in configuration, rather than in application code or during incidents.
- No per-service limiter code. A few lines of Nginx config replace rate-limit logic that each backend team would otherwise write, test, and maintain in its own language and framework.
- Fewer incidents caused by a single client. A retry loop, a misconfigured cron job, or a scraper is cut off at the proxy before it exhausts database connections or worker threads. That avoids the paging, triage, and manual IP blocking that usually follow.
- Safer rollouts.
limit_req_dry_runand$limit_req_statuslogging show which clients would be throttled before enforcement is turned on, so tuning happens from log analysis rather than from rollbacks and support tickets. - Faster client-side fixes. A consistent 429 with
Retry-Afterand a JSON body gives SDK and integration developers a clear signal to implement backoff, instead of debugging intermittent 503s that look like server failures. - Quick tuning loops. Changing
rate,burst, ordelayis a config edit and reload, not a code change and redeploy, which keeps each tuning iteration short.
As an illustrative estimate with assumed inputs: suppose a team has an assumed four incidents per quarter caused by a single runaway client, each assumed to take three engineers about two hours to diagnose and mitigate. That is roughly 24 engineer-hours per quarter. If edge rate limiting prevents most of those incidents and initial setup plus dry-run tuning is assumed to take a day or two, the setup pays for itself within the first quarter. Your own numbers will depend on incident frequency and how much application-level limiting you would otherwise build.
How should you choose the rate limit key and size the shared memory zone?
The key determines who shares a bucket, so a poor key choice can undo careful tuning of rate and burst. $binary_remote_addr is the usual default because it is compact (4 bytes for IPv4, 16 for IPv6) and needs no trust in request content. It also matches clients poorly. An office behind NAT, a corporate proxy, or a mobile carrier's gateway can put hundreds of users behind one address, and they all get throttled together. An attacker spread across many addresses gets a full allowance per IP.
Keying on an API key or user ID tracks the actual consumer. The cost is that Nginx must extract identity at the edge, usually from a header, and an unverified header lets clients pick their own bucket. Requests with an empty key are not counted by that zone, so pair a per-key zone with a per-IP zone for anonymous traffic.
| Key | Accuracy | Trust required | Weakness |
|---|---|---|---|
$binary_remote_addr |
Per network address | None (after realip) | NAT over-limits, botnets under-limit |
| API key header | Per credential | Key must be validated | Unvalidated keys can be spoofed or rotated |
| Authenticated user ID | Per account | Auth must run before limiting | Unauthenticated traffic unlimited by this zone |
Behind a load balancer, every request arrives from the balancer's IP, so all traffic lands in a single bucket. Use the realip module. Set set_real_ip_from to the balancer's address ranges only and real_ip_header to X-Forwarded-For. Add real_ip_recursive on when there are multiple proxy hops.
Size the zone for peak distinct keys. The Nginx documentation estimates roughly 8,000 states per megabyte on 64-bit platforms. When the zone fills, Nginx evicts the least recently used entries. If it still cannot allocate a new state, it rejects the request with an error.
set_real_ip_from 10.0.0.0/16;
set_real_ip_from 192.0.2.0/24;
real_ip_header X-Forwarded-For;
real_ip_recursive on; # skip trusted hops to find the first untrusted address
map $http_x_api_key $api_key_limit {
"" "";
default $http_x_api_key;
}
map $http_x_api_key $anon_ip_limit {
"" $binary_remote_addr;
default "";
}
limit_req_zone $api_key_limit zone=per_key:10m rate=20r/s;
limit_req_zone $anon_ip_limit zone=per_ip:10m rate=5r/s;
limit_req_status 429;
server {
listen 80;
location /api/ {
# Each request is counted by exactly one of these zones.
limit_req zone=per_key burst=40 nodelay;
limit_req zone=per_ip burst=10 nodelay;
proxy_pass
}
}
This config restores the real client IP behind a load balancer, limits requests with an API key per key and anonymous requests per IP, and sizes each shared memory zone for its expected number of distinct keys.
Production config: 429 responses, logging, layered limits, and dry run
Set limit_req_status 429. The default is 503, which tells clients and load balancers that the server is failing rather than that the client is sending too fast. Many HTTP client libraries treat 503 as a reason to retry immediately, which adds load. A 429 tells them to slow down.
Pair it with error_page 429 @ratelimited and a named location that sets default_type application/json and returns a small body such as {"error":"rate_limited"}. Add a Retry-After header with add_header ... always, because without always Nginx does not attach headers to error responses. A clear JSON response lets SDK authors build backoff into their clients instead of reverse-engineering an HTML error page.
limit_req_log_level controls how loud rejections are in the logs. Setting it to warn stops rate-limit events from flooding error-level alerting. Delayed requests are logged one level below rejections.
Layering limits
You can stack several limit_req directives in one context, and every one of them applies. A typical setup is a generous per-IP limit plus a per-API-key limit, with a much stricter zone on /login or expensive report endpoints.
There is one inheritance trap. If a location defines any limit_req, it does not inherit the directives from the server level. Repeat the shared limits in that location explicitly.
Observe before enforcing
Newer Nginx versions support limit_req_dry_run on. In dry-run mode, Nginx counts requests against the limit but does not delay or reject them. Log $limit_req_status, where available, to see REJECTED_DRY_RUN counts per key.
Running in dry-run mode for a week shows you which partner integrations or batch jobs would break before they actually break. That saves the incident, the rollback, and the support threads that follow.
limit_req_zone $binary_remote_addr zone=per_ip:10m rate=20r/s;
limit_req_zone $http_x_api_key zone=per_key:10m rate=50r/s;
limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m;
log_format ratelimit '$remote_addr key=$http_x_api_key $request_uri rl=$limit_req_status';
server {
listen 80;
access_log /var/log/nginx/ratelimit.log ratelimit;
limit_req_status 429; # "slow down", not 503 "server failing"
limit_req_log_level warn; # keep rejections out of error-level alerts
limit_req_dry_run on; # count only; remove once logs look safe
error_page 429 @ratelimited;
# Layered limits: both apply to every request in this context
limit_req zone=per_ip burst=40 nodelay;
limit_req zone=per_key burst=100 nodelay;
location /login {
# Any limit_req here drops the server-level ones, so repeat them
limit_req zone=per_ip burst=40 nodelay;
limit_req zone=per_key burst=100 nodelay;
limit_req zone=login burst=3;
proxy_pass
}
location / {
proxy_pass
}
location @ratelimited {
default_type application/json;
add_header Retry-After 1 always; # always: attach header on error responses
return 429 '{"error":"rate_limited","retry_after_seconds":1}';
}
}
Layered per-IP, per-key, and login limits that return a JSON 429 with Retry-After, run in dry-run mode while $limit_req_status is logged.
How do you test Nginx rate limits safely before enforcing them?
Dry run tells you who would be affected. It does not tell you how the limit feels to a client. For that, point a load generator at a staging instance that runs the same config with enforcement on.
Shape the traffic like a real client, not a flood. A mobile app that opens a screen and fires twelve requests at once is a burst followed by idle time. Reproduce that with something like ab -n 12 -c 12 against a staging endpoint, wait a few seconds, then repeat. A steady -c 50 run for a minute only shows you the sustained rate. It hides whether burst and delay absorb the spikes your clients actually produce.
Then read the error log. Rejections appear as limiting requests and delays as delaying request. Delays are logged one severity level below limit_req_log_level, so error_log must be set at least that verbose to record them. Count each with grep -c 'limiting requests' /var/log/nginx/error.log and grep -c 'delaying request' /var/log/nginx/error.log. If you log $limit_req_status, awk over the access log gives PASSED, DELAYED, and REJECTED counts per key.
Compare those counts with what the client saw. Record status codes and per-request latency from the load tool and check them against your intended shape. For example, the first eight requests should be immediate, the next four spaced out, and none should get a 429. If clients receive 429s that you meant to queue, raise burst. If they get latency you meant to avoid, raise delay. Change one value at a time and rerun the same burst so the results stay comparable.
When should you avoid limit_req, and what are the alternatives?
limit_req is a per-instance control. Its zone lives in shared memory on one Nginx host, so workers on that host share counters but separate hosts do not. Behind a load balancer spreading traffic evenly across four nodes, a 10r/s limit lets a client make roughly 40 requests per second in aggregate. That is fine for smoothing bursts but wrong for enforcing a contractual per-customer quota.
It also does not stop large distributed attacks. When thousands of source IPs each stay under the per-key rate, every request passes. When traffic does get rejected, Nginx still has to accept the connection and parse the request first, so the host's bandwidth and connection capacity remain exposed. Rate settings also need tuning against real traffic. A generous guess never triggers. A tight guess returns 429s to your heaviest legitimate integrations.
| Option | What it controls | Scope | Build and run cost |
|---|---|---|---|
limit_req |
Request rate per key | Single Nginx instance | Low: config only |
limit_conn |
Concurrent connections per key | Single Nginx instance | Low: config only |
| Application or shared-store limiter | Global per-key quotas, tiers, billing-aware limits | All nodes | Higher: code, a shared counter store, and its availability and latency |
| Upstream network protection | Volumetric and distributed floods | Before traffic reaches your hosts | Varies, usually an external service or network capability |
These options are usually layered. Use limit_conn when slow, long-held requests are the problem rather than request rate. Move to a centralized limiter when quotas must be exact across nodes, and accept the cost that comes with it: a store on the hot path and a decision about what happens when that store is unavailable. Keep limit_req as the cheap per-node guard in front of everything else.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to nginx limit_req burst, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.