What is nginx limit_req and how do limit_req_zone and limit_req fit together?
limit_req is nginx's built-in request-rate limiter, provided by the ngx_http_limit_req_module. It caps how fast requests matching a given key are passed through. In an API gateway it does two jobs. It keeps upstream services from being driven past the throughput they can sustain, and it stops a single misbehaving client, such as a retry loop or a scraper, from using capacity that other callers need.
The feature uses two directives with separate roles.
limit_req_zone is declared in the http context and defines the rule:
- The key that groups requests.
$binary_remote_addrlimits per client IP and uses less memory than the text form$remote_addr. A header variable such as$http_x_api_keylimits per API key. A variable derived frommaplets you build tiered keys. Requests whose key evaluates to an empty string are not counted, so amapcan exempt trusted traffic by producing an empty key for it. - The shared memory zone, given as
zone=name:size. It stores per-key state and is shared by all worker processes. If the zone fills up, nginx evicts old entries, and when it cannot free space it rejects the request. - The rate, in
r/sorr/m. User/mfor limits below one request per second.
limit_req applies a named zone in an http, server or location block. With it you can enforce strict limits on expensive endpoints and looser ones elsewhere while sharing a single key definition.
This approach suits teams already running nginx as a gateway or reverse proxy who want per-instance protection without deploying Redis or a separate rate-limiting service. The state is local to each nginx instance, so with N instances behind a load balancer, a client can get up to roughly N times the configured rate.
http {
# Mark trusted internal networks so they can bypass the per-IP limit.
geo $is_external {
default 1;
10.0.0.0/8 0; # internal callers, e.g. batch jobs
}
# Empty key => request is not counted by limit_req.
map $is_external $client_ip_key {
0 "";
1 $binary_remote_addr;
}
# Per-IP rule: 10 requests/second, state shared by all workers.
limit_req_zone $client_ip_key zone=per_ip:10m rate=10r/s;
# Per-API-key rule for an expensive endpoint: 30 requests/minute.
# Requests with no X-API-Key header have an empty key and are not counted here.
limit_req_zone $http_x_api_key zone=per_key:10m rate=30r/m;
# Return 429 instead of the default 503 when a request is rejected.
limit_req_status 429;
upstream api_backend {
server 127.0.0.1:8080;
}
server {
listen 80;
location /api/ {
limit_req zone=per_ip; # baseline protection
proxy_pass
}
location /api/reports/ {
limit_req zone=per_ip; # both limits apply;
limit_req zone=per_key; # the strictest one wins
proxy_pass
}
}
}
Two limit_req_zone rules, one per client IP with internal networks exempted through an empty map key and one per API key, applied by limit_req with a stricter combination on an expensive location.
How does nginx limit_req work as a leaky bucket with a queue?
Nginx does not count requests per second. It converts the configured rate into an interval and checks each request against it with millisecond precision. rate=10r/s means one request every 100 ms, not "any 10 requests within a given second." Internally, nginx tracks how far each key is ahead of that schedule. The excess drains continuously at the configured rate, like water leaking from a bucket.
With no burst, the bucket has zero headroom. A request that arrives before the previous one has drained is rejected immediately, with 503 by default. This is where well-behaved clients get hurt. Suppose a single-page app fires eight parallel API calls on page load within about 20 ms. That is well under 10 r/s averaged over the second, but only the first call passes. The other seven arrive inside the same 100 ms window and are rejected. The client did nothing abusive. It was simply measured at a granularity it never expected.
burst adds a queue in front of the drain. It does not grant extra allowance. Excess requests wait in the queue, up to burst slots, and are released one per interval. Only requests that find the queue full are rejected.
incoming (8 requests in ~20 ms)
│ │ │ │ │ │ │ │
▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼
┌───────────────────────┐
│ burst queue (size=5) │──── queue full ───► 503 (rejected)
│ [r2][r3][r4][r5][r6] │ r7, r8
└──────────┬────────────┘
│ drain: 1 request / 100 ms
▼
upstream r1 now, r2 at +100 ms, r3 at +200 ms ...
The long-run rate stays at 10 r/s. burst only changes whether short spikes are absorbed through queueing or rejected outright. The cost of absorbing them is latency: in this diagram, r6 waits about 500 ms before nginx forwards it.
client (SPA page load): r1..r8 arrive within ~20 ms, same key
│
▼
┌──────────────────────────────────┐
│ limit_req zone (per-key state) │
│ rate=10r/s → 1 slot per 100 ms │
└────────────────┬─────────────────┘
│
▼
┌────────────────────┐ yes
│ on schedule? │─────────► upstream now (r1)
└─────────┬──────────┘
│ no: arrived before previous drained
▼
┌────────────────────┐ no
│ free slot in │─────────► 503 rejected (r7, r8)
│ burst queue? │ with burst=0: r2..r8 land here
└─────────┬──────────┘
│ yes
▼
┌──────────────────────────────────┐
│ burst queue (burst=5) │
│ [r2] [r3] [r4] [r5] [r6] │
└────────────────┬─────────────────┘
│ drain: 1 request / 100 ms
▼
upstream: r2 +100ms r3 +200ms r4 +300ms r5 +400ms r6 +500ms
With rate=10r/s and burst=5, eight near-simultaneous requests yield one immediate pass, five queued requests released one per 100 ms, and two rejections; without burst, seven of the eight would be rejected.
What do burst, nodelay and delay= actually change?
Plain burst holds excess requests in a queue and releases them at the zone rate. The client sees latency instead of an error, but each queued request keeps a client connection open on the worker. If the client's timeout or an upstream deadline is shorter than the queue wait, the request fails anyway after consuming a slot.
nodelay forwards every request that fits within burst immediately. The slots it occupies still drain at the configured rate, so sustained throughput stays capped. The cost moves to the upstream, which must absorb up to burst + 1 requests at the same instant.
delay=N sits between the two. The first N excess requests pass straight through, the rest of the burst allowance is delayed, and anything beyond burst is rejected. This fits clients that send small batches, such as a mobile app firing a handful of calls on screen load. Those batches go through without added latency, while a runaway loop is still slowed and then rejected.
| Mode | Within burst | Beyond burst | Main cost |
|---|---|---|---|
burst only |
Queued, paced | Rejected | Latency, held connections |
nodelay |
Forwarded immediately | Rejected | Upstream spike |
delay=N |
First N immediate, rest paced | Rejected | Tuning N per client type |
Worked example (illustrative estimate)
Assume rate=10r/s, burst=20, and 30 requests from one key arriving at once. The first request is counted as within rate.
burstonly: 1 immediate, 20 delayed (the last waits about 2 s), 9 rejected.nodelay: 21 immediate, 9 rejected.delay=10: 11 immediate, 10 delayed (the last waits about 1 s), 9 rejected.
# http context: one zone per mode so the three examples do not share counters.
# 10m of shared memory tracks per-client state keyed by client IP.
limit_req_zone $binary_remote_addr zone=api_queue:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=api_nodelay:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=api_delay:10m rate=10r/s;
upstream api_backend {
server 127.0.0.1:8080;
}
server {
listen 80;
# Return 429 instead of the default 503 so clients can tell throttling from outages.
limit_req_status 429;
# burst only: up to 20 excess requests are queued and released at 10r/s.
# A 30-request spike: 1 immediate, 20 paced (last waits ~2 s), 9 rejected.
location /v1/queue/ {
limit_req zone=api_queue burst=20;
proxy_pass
}
# nodelay: requests that fit in the burst are forwarded at once,
# but slots still free at 10r/s. Same spike: 21 immediate, 9 rejected.
location /v1/nodelay/ {
limit_req zone=api_nodelay burst=20 nodelay;
proxy_pass
}
# delay=10: first 10 excess requests pass immediately, the next 10 are paced.
# Same spike: 11 immediate, 10 paced (last waits ~1 s), 9 rejected.
# Small batches from a client (e.g. app screen load) see no added latency.
location /v1/delay/ {
limit_req zone=api_delay burst=20 delay=10;
proxy_pass
}
}
Three locations apply the same 10r/s rate with burst=20 in queued, nodelay and delay=10 modes, with comments mapping each to the illustrative 30-request spike outcomes.
How do you implement nginx limit_req for an API gateway?
Start with how rejections look to clients. limit_req_status 429 replaces the default 503, so clients and dashboards can tell throttling apart from an unhealthy upstream. limit_req_log_level warn keeps rejections out of error-level alerting, and delayed requests are logged one level below that. Point error_page 429 at a named location that sets default_type application/json and returns a small JSON body, so API clients get something they can parse. Nginx does not send Retry-After or rate-limit headers itself. Any you add with add_header ... always are static guesses, not values computed from the bucket's state.
Key choice decides who shares a bucket. Behind a load balancer, $remote_addr is the balancer's address. Configure the realip module first: set_real_ip_from for trusted proxy ranges, real_ip_header X-Forwarded-For, and real_ip_recursive on. Then key on $binary_remote_addr, which is smaller in shared memory than the text form. Clients behind carrier NAT still share one address, so for authenticated APIs key on the credential, for example $http_x_api_key. Use map to translate internal callers' keys to an empty string. Requests with an empty key are not counted against that zone.
You can stack several limit_req directives in one block, such as a per-key zone plus a per-endpoint zone keyed on a constant. A request must pass every one of them. Inheritance is all-or-nothing: a location inherits the server's limit_req directives only if it defines none of its own. Adding one limit to /login silently drops the server-wide limits unless you repeat them in that location.
http {
# Trust X-Forwarded-For only from our load balancers, so $binary_remote_addr is the client
set_real_ip_from 10.0.0.0/8;
real_ip_header X-Forwarded-For;
real_ip_recursive on;
# Internal callers map to "" and are not counted. Requests with no API key fall back to client IP.
map $http_x_api_key $api_limit_key {
default $http_x_api_key;
"" $binary_remote_addr;
"internal-batch-key-example" "";
}
limit_req_zone $api_limit_key zone=per_key:10m rate=10r/s;
limit_req_zone $server_name zone=search_global:1m rate=200r/s; # one bucket per server
limit_req_zone $binary_remote_addr zone=login_ip:10m rate=1r/m;
limit_req_status 429; # instead of 503, so throttling is distinct from upstream failure
limit_req_log_level warn; # rejections at warn, delayed requests at notice
server {
listen 80;
server_name api.example.com;
error_page 429 @rate_limited;
limit_req zone=per_key burst=20 nodelay; # inherited only by locations with no limit_req
location /v1/search {
# Defines its own limit_req, so the server-level one must be repeated here
limit_req zone=per_key burst=20 nodelay;
limit_req zone=search_global burst=100; # must pass both
proxy_pass
}
location /v1/login {
limit_req zone=login_ip burst=5 nodelay; # per_key no longer applies here
proxy_pass
}
location @rate_limited {
default_type application/json;
add_header Retry-After 1 always; # static guess, not derived from bucket state
return 429 '{"error":"rate_limited","message":"Too many requests"}';
}
}
}
An API gateway config that restores client IPs via realip, exempts internal API keys through map, stacks per-key and per-endpoint limits, and returns a JSON 429. It also shows that a location defining any limit_req drops the server-level limits.
How can you test limit_req before enforcing it?
Turn on limit_req_dry_run on; in the location you plan to protect, where your nginx build supports it. In dry-run mode nginx still runs every request through the bucket and updates zone state, but it never delays or rejects anything. This lets you see how your chosen rate and burst would treat real production traffic without any client seeing an error.
The most useful signal is the $limit_req_status variable. Add it to your access log format. It records PASSED, DELAYED, REJECTED, or the dry-run variants DELAYED_DRY_RUN and REJECTED_DRY_RUN. With that field you can group by client key and route and find out which consumers would be throttled, and how often.
Then generate bursts on purpose. Run a tight curl loop from one host so every request maps to the same key, or use a simple load tool with a fixed request rate. Keep the rate just above your configured rate, then well above rate + burst. Check that the delayed and rejected counts match what the leaky-bucket arithmetic predicts.
The error log separates the two outcomes. Rejections are logged at limit_req_log_level, which defaults to error. Delays are logged one level lower, and both entries report the excess and the zone name. If you want to see delays, make sure your error_log level includes them.
Most of the engineering time saved comes from this step. Without it, tuning happens during incidents: a partner gets 503s, someone raises burst under pressure, and the real cause goes unexamined. A dry-run pass gives you evidence before enforcement, so you change the config once instead of reacting to each complaint.
When should you avoid nginx limit_req?
limit_req is a good local guard, but several of its properties make it the wrong primary control for some APIs.
State is per instance. Each gateway keeps its own shared memory zone. Behind a load balancer with N nodes, a client can get roughly N times the configured rate. If balancing is uneven, for example because of sticky sessions or long-lived keepalive connections, the effective limit varies from client to client. If you need a contractual quota per customer across the fleet, the counter must live somewhere shared.
IP keys are a poor identity. Hundreds of users behind a corporate NAT or a mobile carrier gateway share one address and one bucket, so they throttle each other. Meanwhile, a distributed client spreads its traffic across many addresses and stays under every per-IP limit.
Keys can attack the zone. If the key is attacker-controlled and high-cardinality, such as a header or an unvalidated token, an attacker can generate fresh keys to fill the zone. Nginx evicts least-recently-used entries to make room. That resets the buckets of real clients, and when no space can be freed, requests with new keys are rejected.
Every request costs the same. A cheap GET and an expensive export or search consume one slot each. Separate locations help, but nginx cannot weight requests by actual cost.
Burst is a blunt trade-off. A large burst lets short attacks pass through untouched. A small one rejects legitimate clients during normal spikes.
If you need global quotas, tenant-aware identity, or cost-based limits, enforce them in a shared limiter or in the application. Keep limit_req as a coarse per-node backstop.
What are the alternatives to limit_req and what do they cost to run?
Each alternative targets a different failure mode, and everything beyond limit_req and limit_conn adds something your team has to build, deploy, and page on.
| Mechanism | What it limits | Development cost | Operational cost |
|---|---|---|---|
limit_req |
Request rate per key, per node | Configuration only | No extra services |
limit_conn |
Concurrent connections per key, per node | Configuration only | No extra services |
| Shared-state limiter | Global quota across all nodes | Store or service integration, key design | A dependency on the hot path, added latency, a fail-open or fail-closed decision, more on-call surface |
| Application-level limits | Per-endpoint or per-tenant cost | Code in each service, plus a cost model | Tuning, tests, and releases tied to app deploys |
| Upstream queues and timeouts | Work admitted to the backend | Server and pool settings | Tuning, but no early rejection at the edge |
limit_req counters live in each node's shared memory, so the limits are not global.
Illustrative estimate (assumed numbers, not a benchmark): effective client rate ≈ per-node rate × nodes that receive that client's traffic. Assume rate=10r/s and 4 nodes behind round-robin. One API key could reach about 10 × 4 = 40 r/s. With uneven balancing, the real figure falls somewhere between 10 and 40 r/s.
Stay with limit_req when your goal is to protect nodes from floods, your keys are coarse (such as IP address or API key), and approximate per-node fairness is acceptable. Move to a shared limiter when quotas are contractual or billed, or when node counts change often. Move to application limits when request cost varies widely by endpoint.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to nginx limit_req, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.