What is nginx limit_req and how does the leaky bucket work?
limit_req (from ngx_http_limit_req_module) implements a leaky bucket. It does not count requests per second and reset the count at each second boundary. It tracks how quickly requests arrive and lets them through at a steady rate. Requests pour into the bucket, the bucket drains toward your upstream at a fixed rate, and anything that would overflow it gets rejected.
client requests (bursty)
│ │ │││ │ │
▼ ▼ ▼▼▼ ▼ ▼
┌──────────────────┐
│ [ ][ ][ ][ ][ ] │ ← burst slots (queue of excess requests)
└────────┬─────────┘ overflow → rejected (503 by default)
│
▼ drains at the configured rate, e.g. 1 every 100ms
┌──────────┐
│ upstream │
└──────────┘
nginx enforces the rate at millisecond granularity. A rate of 10r/s means "one request every 100ms," not "any ten requests during a given second." Suppose five requests from the same key arrive in the same instant and no burst is configured. nginx forwards the first one and rejects the other four, because each of them arrived less than 100ms after the previous request it accepted. Engineers who expect a per-second counter tend to see this as a bug. It is the defined behavior, and the burst parameter exists to handle it.
Teams use limit_req because smoothing protects the services behind nginx. A traffic spike gets flattened into a rate your application servers, database pools, or third-party APIs can sustain. Rejection is cheap and happens at the edge, so it also blunts basic abuse such as credential stuffing against a login endpoint or a scraper hammering a search route. That protection kicks in before any upstream work is done.
http {
# Shared memory zone keyed by client IP.
# 10m holds state for many thousands of addresses; rate 10r/s = one request per 100ms.
limit_req_zone $binary_remote_addr zone=per_ip:10m rate=10r/s;
upstream app_backend {
server 10.0.0.11:8080;
server 10.0.0.12:8080;
}
server {
listen 80;
server_name api.example.com;
location /login {
# No burst: five simultaneous requests from one IP -> first is
# forwarded, the other four are rejected immediately, because each
# arrives less than 100ms after the last accepted request.
limit_req zone=per_ip;
# 503 is the default rejection status; 429 tells clients to back off.
limit_req_status 429;
proxy_pass
}
location /search {
# Up to 5 excess requests wait in the bucket's burst slots and are
# drained to the upstream at 1 per 100ms; the 6th excess is rejected.
limit_req zone=per_ip burst=5;
limit_req_status 429;
proxy_pass
}
}
}
A per-IP limit_req zone at 10r/s, where /login rejects any request arriving within 100ms of the last accepted one and /search queues up to five excess requests that drain to the upstream at the configured rate.
How does limit_req work on each request?
The bucket picture is a model. Mechanically, nginx follows the same short sequence for every request that reaches a location with limit_req configured:
- Evaluate the key. nginx evaluates the key expression from
limit_req_zone, for example$binary_remote_addr. If the result is empty, the request is not counted and passes on to the next limit, or through. - Look up the state. The key is hashed and looked up in the named shared-memory zone. If no state exists yet, nginx allocates one. If it cannot, it first evicts least recently used states, and if allocation still fails it rejects the request.
- Update the excess. Each state records how far over the rate the key currently is (the "excess") and when it was last updated. nginx subtracts what the bucket has drained since that time at the configured rate, never going below zero, and then adds one for the current request.
- Decide. If the new excess exceeds
burst, the request is rejected withlimit_req_status(503 by default) and the stored state is left unchanged. If the request is within burst but above the immediate-pass threshold, it is delayed until its slot comes up. The threshold is zero withburstalone, the full burst withnodelay, and N withdelay=N. Otherwise it continues immediately. - Record the outcome. Rejections and delays are written to the error log at the configured
limit_req_log_level. The outcome is also exposed in$limit_req_statusfor access logging.
All of this happens in nginx's preaccess phase, before access checks, authentication subrequests and proxying. A rejected request therefore costs a hash lookup and a small response, not an upstream connection. Because the state lives in shared memory, every worker process on the same instance sees and updates the same buckets. The state is not shared with other nginx instances, and the consequences of that are covered below.
How do you define a rate limit zone and choose its key?
Every limit_req_zone directive takes three parts: a key, a named shared-memory zone with a size (zone=api:10m), and a rate (rate=10r/s, or r/m for slower limits). The key decides who shares a bucket. The zone stores one state per distinct key value, and every worker process reads and writes that same zone.
Use $binary_remote_addr rather than $remote_addr for per-IP limits. The binary form is a fixed 4 bytes for IPv4 and 16 for IPv6. The text form can run to 15 or 39 bytes. Smaller keys mean more states fit in the same zone. The nginx documentation estimates that a 1 MB zone holds roughly 8,000 states on 64-bit platforms. Treat that figure as a starting point for sizing, not a guarantee.
| Use case | Key | Caveat |
|---|---|---|
| Per client IP | $binary_remote_addr |
NAT and proxies merge many users into one IP |
| Per API key | $http_x_api_key |
Requests without the header produce an empty key |
| Per user | variable set from auth, e.g. via map |
Must be resolved before the limit applies |
| Exempt trusted clients | map that outputs "" for them |
Empty keys are never counted |
Requests with an empty key are not rate limited at all, which makes the exemption pattern dangerous. If your map default is "", or the header it reads is absent, then every client is exempt. Make the limited value the default, and set the empty value only for explicit matches.
When the zone fills, nginx evicts the least recently used states. If it still cannot allocate a state, it rejects the request. An undersized zone therefore fails closed and blocks legitimate traffic under load. An oversized zone only wastes shared memory. For that reason, it is safer to err toward a larger zone.
http {
# Mark trusted internal ranges (placeholder CIDRs).
geo $trusted_client {
default 0;
10.0.0.0/8 1;
192.168.0.0/16 1;
}
# Exemption pattern: the DEFAULT is the limited value.
# Only explicit trusted matches get an empty key, which is never counted.
map $trusted_client $ip_limit_key {
default $binary_remote_addr;
1 "";
}
# Per-API-key limiting. A missing header would yield an empty key
# (unlimited), so fall back to the client IP instead.
map $http_x_api_key $api_limit_key {
default $http_x_api_key;
"" $binary_remote_addr;
}
# key, zone name:size, rate
limit_req_zone $ip_limit_key zone=per_ip:10m rate=10r/s;
limit_req_zone $api_limit_key zone=per_key:20m rate=300r/m;
limit_req_status 429;
server {
listen 80;
location /api/ {
# Both limits apply; the stricter one decides.
limit_req zone=per_ip burst=20;
limit_req zone=per_key burst=50;
proxy_pass
}
}
}
Two limit_req_zone definitions, one keyed per IP with a map-based exemption for trusted ranges and one keyed per API key with an IP fallback, both written so that the default path produces a non-empty, limited key.
What do burst, nodelay and delay actually change?
Every request above the configured rate falls into the burst queue or is rejected. The burst, nodelay and delay parameters control what happens inside that queue.
burst alone queues excess requests and releases them at the base rate. Illustrative estimate: with rate=10r/s and burst=20, nginx releases one request every 100 ms. If 21 requests arrive together, the first is served, 20 wait, and the last waits about 2 seconds. Request 22 is rejected, with 503 by default.
burst with nodelay forwards all queued requests immediately. Their slots still drain at 10r/s, though, so after the spike a client gets one new request every 100 ms. This differs from raising the rate to 30r/s. Raising the rate permits a sustained 30r/s, while nodelay permits a one-off spike and then holds the client to the original average.
delay=N applies two stages. Take rate=10r/s burst=20 delay=8 with 25 simultaneous requests:
- Request 1 is within the rate and is served.
- Requests 2–9 are the first 8 excess requests and are served immediately.
- Requests 10–21 are delayed from roughly 100 ms up to about 1.2 s.
- Requests 22–25 exceed the burst and are rejected.
| Mode | Client latency | Upstream load during spike | Rejection |
|---|---|---|---|
burst |
Grows with queue position | Smoothed to the base rate | Beyond burst |
burst nodelay |
None | Full spike at once | Beyond burst |
burst delay=N |
None for the first N, then grows | Partial spike, then smoothed | Beyond burst |
Queuing keeps connections and worker slots open while clients wait. nodelay keeps latency low but sends the whole spike to the backend. delay=N lets you size that spike to what the upstream can handle.
limit_req_zone $binary_remote_addr zone=perip:10m rate=10r/s;
upstream app_backend {
server 127.0.0.1:8080;
}
server {
listen 80;
# Rejected requests get 503 by default; uncomment to return 429 instead.
# limit_req_status 429;
# burst alone: up to 20 excess requests wait in the queue and are
# released at 10r/s. The 20th queued request waits about 2 s.
location /reports/ {
limit_req zone=perip burst=20;
proxy_pass
}
# burst + nodelay: all 20 excess requests are forwarded at once,
# but their slots still free up at 10r/s, so the upstream sees the full spike.
location /static-api/ {
limit_req zone=perip burst=20 nodelay;
proxy_pass
}
# burst + delay=8: the first 8 excess requests go through immediately,
# the next 12 are paced at 10r/s, and anything beyond burst=20 is rejected.
location /api/ {
limit_req zone=perip burst=20 delay=8;
proxy_pass
}
}
One per-IP zone applied three ways: queued with burst alone, forwarded immediately with nodelay, and split into an immediate stage and a paced stage with delay=8 (the delay parameter requires nginx 1.15.7 or later).
How do multiple limit_req directives combine and inherit?
You can declare several limit_req directives in the same context, and nginx checks every one of them for each request. A common pairing is a per-client zone keyed on $binary_remote_addr plus a per-server zone keyed on $server_name. The per-IP zone stops one noisy client. The per-server zone caps aggregate load on the backend when many clients are each well-behaved but collectively too much. If any applicable limit is exceeded with no burst room left, the request is rejected. If more than one limit wants to delay the request, the longest delay wins. The effective limit is always the strictest one that applies.
Inheritance is the part that catches people. limit_req directives are inherited from the enclosing http or server block only if the current block defines none of its own. A location that adds a single limit_req for a tighter per-endpoint rule drops the server-level per-IP and global limits entirely. If you want them to keep applying, repeat them inside that location.
The same rule affects exemptions. There is no off switch, so a health-check location under a rate-limited server still inherits the server's limits. You have two options:
- Attach limits only at the
locationlevel. - Use a
mapthat yields an empty key for exempt paths. Requests with an empty key are not counted.
Splitting locations matters because limit_req counts requests, not cost. A /healthz probe and a full-text /search query each consume one slot. Put search in its own location with its own low-rate zone. Keep cheap endpoints in a separate location, either unlimited or generously limited, so that probes and static fetches don't use up the allowance meant to protect expensive work.
http {
# Empty key for /healthz means probes are never counted in these zones.
map $uri $ip_key { /healthz ""; default $binary_remote_addr; }
map $uri $srv_key { /healthz ""; default $server_name; }
limit_req_zone $ip_key zone=perip:10m rate=10r/s; # one noisy client
limit_req_zone $srv_key zone=perserver:1m rate=200r/s; # aggregate backend cap
limit_req_zone $binary_remote_addr zone=search:10m rate=1r/s; # expensive endpoint
limit_req_status 429;
server {
listen 80;
server_name api.example.com;
# Both limits are checked on every request; the strictest one decides.
limit_req zone=perip burst=20 nodelay;
limit_req zone=perserver burst=400 nodelay;
location / {
# Defines no limit_req, so it inherits both server-level limits.
proxy_pass
}
location /search {
# Defining any limit_req here drops the inherited ones,
# so the server-level limits are repeated explicitly.
limit_req zone=search burst=5;
limit_req zone=perip burst=20 nodelay;
limit_req zone=perserver burst=400 nodelay;
proxy_pass
}
location = /healthz {
# Inherits the server limits, but the map gives it an empty key,
# so these requests are not counted.
proxy_pass
}
}
}
Stacked per-IP and per-server limits at the server level, a /search location that repeats them alongside its own low-rate zone because defining limit_req there drops the inherited ones, and a map that gives /healthz an empty key so probes are not counted.
How do you roll out, test and observe nginx rate limits safely?
Set limit_req_status 429 before anything else. The default is 503, which clients, retry libraries and dashboards read as "server broken" rather than "slow down". A 429 tells well-behaved clients to back off, and it keeps rate-limit rejections out of your 5xx error-rate alerts.
For a new limit, start with limit_req_dry_run on. Nginx still runs the leaky-bucket accounting and logs what it would have done, but every request passes through. Leave it on long enough to cover real peaks, such as a deploy window or a nightly batch job. That shows whether your rate and burst would hit legitimate traffic before any client sees an error.
limit_req_log_level sets the severity of rejection entries. Delayed requests are logged one level lower. With the default error, rejections log as error and delays as warn. Lines starting "limiting requests" are rejections. Lines starting "delaying request" were queued. Dry-run lines say so explicitly. The excess field shows how far over the configured rate the bucket was. If it often sits just under your burst value, the limit is close to biting. For per-request analysis, add the $limit_req_status variable to your access log format.
To test, a sequential curl loop that prints only status codes is enough for low rates. A sequential loop can be slower than the limit, so use a concurrent tool such as ab to actually exceed it.
Development cost is low because the work is only configuration. The real investment is tuning the limits and watching the logs afterwards.
How can limit_req save engineering time?
The main saving is that rate limiting becomes one piece of edge configuration instead of code repeated in every service.
- No per-service limiter code. Without an edge limit, each application team has to write, test and maintain its own throttling in its own language and framework, often with subtly different semantics. A
limit_reqzone in front of those services gives them a common baseline. Application code is then needed only for limits that depend on business identity, such as per-tenant quotas. - Changes ship as a config reload, not a deploy. Adjusting a rate or burst means editing the config, validating it with
nginx -t, and reloading withnginx -s reloador your service manager. Nothing is rebuilt, and no application release is involved. During an incident, tightening a limit on one abused endpoint can be done in the time it takes to edit and reload. - Less firefighting on overloaded upstreams. Smoothing spikes before they reach a fragile backend, such as a database-heavy endpoint or a legacy service, removes a class of pages where the fix would otherwise be a manual scale-up or a restart.
- Cheaper, safer rollouts.
limit_req_dry_runand the error-log entries let you validate a limit against real traffic without a separate test environment or a custom traffic replay. Mistuned limits are caught before they block users, which avoids the debugging session that follows a bad rollout. - Faster diagnosis. Because rejections carry their own status code (429 once configured) and log lines name the zone and the excess, it is quick to tell "we throttled this client" apart from "the backend failed."
Illustrative estimate, with assumed inputs: assume an organisation has 6 services that each need basic per-IP throttling, and that writing, reviewing and testing an in-app limiter takes an assumed 2 engineer-days per service. Building them all would cost about 12 engineer-days, plus ongoing maintenance in 6 codebases. If one engineer spends an assumed 2 days writing and dry-running equivalent limit_req rules at the edge, the upfront difference is roughly 10 engineer-days. These figures are assumptions for the arithmetic, not measurements. Your own numbers will depend on how much the services' limiting needs actually overlap.
The saving has limits. Edge rules do not replace identity-aware quotas, and in multi-instance deployments you still have to reason about per-node behaviour, as described below.
How do you get the real client IP behind a load balancer or CDN?
When Nginx sits behind a load balancer, reverse proxy or CDN, every TCP connection it accepts comes from that intermediary. $binary_remote_addr then holds the proxy's address, so a zone keyed on it places all clients in a single bucket. One busy user can exhaust the limit for everyone, and the rejections look random because they track total traffic, not individual behaviour.
The realip module fixes this by rewriting the client address before limit_req evaluates the key. Three directives control it:
set_real_ip_fromlists the addresses or CIDR ranges you trust to supply a forwarded address, such as your load balancer subnet or your CDN's published egress ranges.real_ip_headernames the header to read. This is usuallyX-Forwarded-FororX-Real-IP, orproxy_protocolif the balancer uses the PROXY protocol.real_ip_recursive onwalksX-Forwarded-Forfrom right to left and selects the first address that is not in a trusted range. With it off, Nginx takes the last entry. In a multi-hop chain such as CDN → load balancer → Nginx, the last entry is often another proxy.
Once the module is configured, $remote_addr and $binary_remote_addr both reflect the client. The original peer address remains available in $realip_remote_addr, which is useful in logs. The module is not compiled in by default, so check nginx -V for it.
Keep the trusted list narrow. Trusting 0.0.0.0/0 lets any client send its own X-Forwarded-For. An attacker can then rotate fake addresses to get a fresh bucket on every request, or put a victim's IP in the header so that the victim gets throttled.
Who should use limit_req?
limit_req is a good fit for:
- Teams that already run nginx in front of their services as a reverse proxy or API entry point. The module is part of the standard build, so adding a limit needs no new component.
- Operators of a single instance or a small, fixed set of instances, where per-instance state gives a predictable limit per key.
- Owners of fragile or expensive upstreams, such as legacy services, database-heavy endpoints, or routes that call rate-limited third-party APIs. Those services benefit from having spikes smoothed before the traffic reaches them.
- Anyone needing coarse abuse protection on specific routes like login, password reset, signup or search. Rejecting excess requests at the edge is cheaper than handling them in application code.
- Platform or SRE teams that want a common baseline limit across several services without asking each team to implement its own.
It is a weaker fit for teams that need exact cluster-wide limits across an autoscaling fleet, billing-grade per-customer quotas, or limits based on request cost rather than request count. The next section covers those cases and the alternatives.
When should you avoid limit_req, and what are the alternatives?
limit_req works best when one nginx instance, or a small fixed number of them, sees all the traffic for a route. It suits coarse protection such as capping login attempts or scraping bursts per client, and it smooths traffic to an upstream that falls over under spikes, like a legacy service or a database-heavy endpoint. In those cases its local, in-memory state is an advantage. There is no network hop and no external dependency, and the decision is made before the request reaches your application.
The model breaks down as you scale out. Each zone lives in one instance's shared memory, and nginx does not synchronise it across machines. Behind a round-robin balancer, ten nodes each enforcing 10 r/s allow roughly 100 r/s per key. The effective limit changes whenever autoscaling adds or removes nodes. Per-IP keys also misfire. Many users behind one office NAT or carrier-grade NAT share an address and a bucket. Meanwhile, a client rotating through proxy or cloud IPs gets a fresh bucket on every hop.
Use these alternatives to supplement or replace it:
| Alternative | Scope | State | Precision |
|---|---|---|---|
limit_conn |
Concurrent connections per key, per instance | Local shared memory | Exact per node; limits concurrency, not request rate |
| Application-level quotas | Per user, API key or tenant | Application database or store | High; tied to business identity |
| Shared-state limiter (app or gateway) | Cluster-wide | Central store | High, at the cost of a network round trip and a dependency |
| Auth-aware abuse detection | Accounts, tokens, behaviour | Logs, event streams | Highest for abuse; slower to react |
| Upstream capacity planning | Whole service | None at request time | Not a limiter; reduces how much limiting you need |
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to nginx limit_req, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.