Monolith Rate Limiting per Endpoint: In-Process vs Redis Limits

What per-endpoint rate limiting is and how to key it

Monolith rate limiting per endpoint means each route gets its own request budget, enforced inside the application rather than as one global ceiling. Teams use it for two reasons. The first is to protect expensive routes, such as a report export or a full-text search, from exhausting database connections or CPU. The second is to enforce fair use, so one client cannot consume capacity that others need.

A limit is only as good as its key. The usual key is the normalized route template plus a client identity, for example GET /users/{id} combined with an API key. Use the template your router matched, not the raw path. Keying on /users/123 and /users/124 separately creates one counter per ID. That inflates memory or Redis keyspace without bound, and a client can sidestep the limit by varying the path.

For identity, prefer an authenticated API key or user ID. Fall back to IP for anonymous traffic, but do not trust X-Forwarded-For directly, because any client can set it. Take the address appended by your own trusted proxy and ignore entries that clients supplied.

Keep policies in config, not scattered through handlers. A cheap GET /products/{id} might allow hundreds of requests per minute while POST /exports allows a handful. Keeping these in one place makes the budgets reviewable and adjustable without code changes.

Per-endpoint limits complement edge or gateway protection. They do not replace it. By the time a request reaches your monolith, a volumetric flood has already consumed bandwidth and connections that only upstream defenses can absorb.

CopyPOLICIES = {
    ("GET", "/products/{id}"): {"limit": 600, "window_s": 60},  # cheap read
    ("GET", "/search"):        {"limit": 60,  "window_s": 60},  # full-text, CPU heavy
    ("POST", "/exports"):      {"limit": 5,   "window_s": 60},  # expensive report
}
DEFAULT_POLICY = {"limit": 300, "window_s": 60}

TRUSTED_PROXIES = {"10.0.0.5"}

def client_ip(remote_addr: str, xff_header: str | None) -> str:
    # If the direct peer is not our proxy, ignore XFF: the client could have set it.
    if remote_addr not in TRUSTED_PROXIES or not xff_header:
        return remote_addr
    # Our single proxy appends the real peer address last. Earlier entries
    # came from the client and are not trusted.
    return xff_header.split(",")[-1].strip()

def rate_limit_key(method: str, route_template: str, api_key: str | None,
                   user_id: str | None, remote_addr: str,
                   xff_header: str | None) -> tuple[str, dict]:
    # Prefer authenticated identity. Fall back to IP only for anonymous traffic.
    if api_key:
        who = f"key:{api_key}"
    elif user_id:
        who = f"user:{user_id}"
    else:
        who = f"ip:{client_ip(remote_addr, xff_header)}"
    policy = POLICIES.get((method, route_template), DEFAULT_POLICY)
    # Template, not raw path: /users/123 and /users/124 share one counter.
    return f"rl:{method}:{route_template}:{who}", policy

key, policy = rate_limit_key("GET", "/users/{id}", "ak_test_placeholder",
                             None, "10.0.0.5", "203.0.113.9, 198.51.100.7")
print(key, policy)  # rl:GET:/users/{id}:key:ak_test_placeholder {...}

A framework-agnostic sketch that keeps per-route budgets in one config map and builds limiter keys from the matched route template plus an API key, user ID or proxy-verified client IP.

How it works: the path of a rate-limited request

Per-endpoint limiting runs as middleware, or as a filter or interceptor, depending on your framework. It must run after routing, because it needs the matched route template. It should also run after authentication when limits are keyed on API key or user ID. For each request it does the following:

1. Resolve the route. The router matches the raw path to a template such as GET /users/{id}.
2. Identify the client. Use the API key or user ID if the request is authenticated. Otherwise use the proxy-verified client IP.
3. Look up the policy. Find the limit and window for that method and template, or fall back to a default.
4. Build the key. Combine method, template and client identity into one limiter key.
5. Check and record atomically. Ask the limiter, in-process or Redis, whether this key has budget left. If it does, record the request in the same operation.
6. Allow or reject. If allowed, the handler runs as normal. If rejected, return 429 Too Many Requests with a Retry-After header without touching the handler, the database or downstream services.

Rejection has to be cheap for this to protect anything. The limiter check costs a map lookup or one Redis round trip. The work it prevents, such as a query, a report render or an outbound call, costs far more. The rest of this article covers the two design choices that matter in steps 5 and 6: which counting algorithm to use, and where the counter state lives.

When to use per-endpoint rate limits

Use per-endpoint limits when routes in the same application have very different costs or risks, and one shared budget would be too loose for some and too tight for others. Typical cases:

• Expensive routes such as exports, report generation, full-text search or bulk endpoints. A few of these can exhaust a connection pool or CPU while cheap reads remain harmless.
• Abuse-prone routes such as login, password reset, OTP verification and signup. Here a low limit per user or IP slows brute-force and enumeration attacks.
• Routes that call metered or capacity-limited dependencies, such as a paid third-party API or a downstream service with a hard ceiling. Your limit must stay below theirs.
• Published or contractual quotas, where customers are promised a specific number of requests per endpoint or per plan.
• Multi-tenant fairness, where one noisy tenant on a hot route should not degrade the route for everyone else.

They add little when every route costs roughly the same and a single global or per-client limit already protects the system. They are also the wrong tool for absorbing volumetric floods, which must be stopped upstream before they reach the application.

How the algorithms differ: fixed window, sliding window and token bucket

Each algorithm trades accuracy against the state it keeps per key, and that state is what you pay for in memory or in Redis round trips.

Fixed window keeps one counter per key per window, for example POST /login per user per minute, and resets it at the boundary. A client can spend the full limit at 0:59 and again at 1:00, which gets close to 2x the limit through within a couple of seconds.

Sliding window log stores a timestamp for every accepted request and counts the ones inside the trailing window. It is exact, but memory grows with the limit: a 1,000-per-minute key can hold 1,000 timestamps.

Sliding window counter keeps the current and previous fixed-window counts. It estimates usage as the current count plus the previous count weighted by how much of the previous window still overlaps. This assumes the previous window's requests were spread evenly. When they were clustered, the estimate can be slightly too strict or too loose near the edges.

Token bucket stores a token count and a last-refill time. Requests spend tokens, and tokens refill at a steady rate. That allows short bursts up to the bucket's capacity, then holds the client to the refill rate.

The in-memory rate limiter: when it is the right default

An in-process limiter keeps state in a concurrent map from key (for example route:client_id) to window state. For a sliding window counter, that state is the current window's count, the previous window's count, and the window start. Each request locks the entry, or uses a compare-and-swap on packed counters, so two goroutines or threads cannot both read 99 and both write 100. A per-key lock beats one global mutex because hot endpoints don't serialize unrelated keys. Keys must expire through a TTL sweep or LRU eviction. Otherwise every distinct API key or IP becomes a permanent entry, and a scan from rotating addresses turns into a memory leak.

The implementation should take a clock as a dependency, either a now() function or an interface, instead of reading system time directly. Tests can then advance time to window boundaries and check the weighted estimate, prev * (1 - elapsed/window) + curr, without sleeping.

This is the right default for a single instance, or wherever the limit protects the process itself rather than enforcing a contract. CPU-heavy report endpoints are a typical case, where approximate limits are acceptable. Avoid it for login throttling, billing quotas, or published per-customer limits, for these reasons:

• The effective limit is the configured limit × replica count. Four replicas at 100/min allow 400/min.
• Dividing the budget per replica (25 each) only holds if the load balancer spreads each client evenly. Sticky sessions or connection reuse break that assumption.
• Autoscaling changes the limit whenever the replica count changes.
• Every deploy or restart resets all counters, which hands attackers a fresh budget.

Copyimport threading, time

class SlidingWindowLimiter:
    def __init__(self, limit, window_s, clock=time.monotonic):
        # clock is injected so tests can jump to window boundaries without sleeping
        self.limit, self.window, self.clock = limit, window_s, clock
        self._entries = {}  # key -> [lock, window_start, prev_count, curr_count]
        self._map_lock = threading.Lock()  # held only to find/create an entry

    def allow(self, key):
        with self._map_lock:
            e = self._entries.setdefault(key, [threading.Lock(), 0.0, 0, 0])
        with e[0]:  # per-key lock: hot routes don't serialize unrelated keys
            now = self.clock()
            start = now - (now % self.window)
            if start != e[1]:
                # Roll over. Previous count only matters if that window was adjacent.
                e[2] = e[3] if start - e[1] == self.window else 0
                e[1], e[3] = start, 0
            elapsed = (now - start) / self.window
            if e[2] * (1 - elapsed) + e[3] >= self.limit:
                return False
            e[3] += 1
            return True

    def sweep(self):
        # Run periodically. Drops keys idle for 2+ windows so rotating IPs can't grow the map forever.
        cutoff = self.clock() - 2 * self.window
        with self._map_lock:
            for k in [k for k, e in self._entries.items() if e[1] < cutoff]:
                del self._entries[k]

t = [60.0]
lim = SlidingWindowLimiter(10, 60, clock=lambda: t[0])
assert all(lim.allow("reports:client_42") for _ in range(10))
assert not lim.allow("reports:client_42")
t[0] = 150.0  # halfway into the next window: estimate = 10 * 0.5 + 0 = 5
assert sum(lim.allow("reports:client_42") for _ in range(10)) == 5

A per-process sliding window counter with per-key locks, a TTL sweep to bound key growth, and an injectable clock that lets a test check the weighted estimate at a window boundary without sleeping.

The Redis rate limiter: atomic shared budgets across replicas

Those drawbacks matter when the limit protects something shared: a paid API quota, a downstream service with a hard ceiling, or a per-customer contract. If exceeding the number costs money or breaks a partner, every replica must draw from one budget, and a network hop per request is a fair price for that.

The naive version reads the counter, compares it in application code, then writes it back. Two replicas can both read 99 against a limit of 100, both allow the request, and both write 100. Under load these interleavings are routine, and the limit drifts upward. The check and the increment have to happen as one operation on the server.

Implementation options

• Fixed window: INCR on a key that includes the window number, plus EXPIRE when the count is 1. INCR returns the new value, so the check is atomic. Send the EXPIRE in the same transaction or script, or a crash between the two commands leaves a key that never expires.
• Sliding window log: a sorted set scored by timestamp. ZREMRANGEBYSCORE removes old entries, ZCARD counts the rest, and ZADD records the request. This is exact, but memory grows with the limit.
• Lua script: runs the check and the increment atomically for any algorithm, including token bucket.

Read the time with TIME inside the script instead of passing it in from each replica. Clock skew between replicas then cannot distort the window. The trade-off is that every limit now depends on one primary's clock and availability.

In a cluster, one hot tenant's key puts all of its load on a single shard. Multi-key scripts also require every key to be in the same hash slot. Use a shared hash tag such as {tenant:42} to keep them together.

Copyimport uuid
import redis

SLIDING_WINDOW = """
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local window_ms = tonumber(ARGV[2])

-- Use the Redis server clock so replica clock skew cannot move the window.
local t = redis.call('TIME')
local now_ms = tonumber(t[1]) * 1000 + math.floor(tonumber(t[2]) / 1000)

redis.call('ZREMRANGEBYSCORE', key, '-inf', now_ms - window_ms)
if redis.call('ZCARD', key) >= limit then
  return 0
end

-- The member must be unique, or two requests in the same millisecond collapse into one.
redis.call('ZADD', key, now_ms, t[1] .. t[2] .. '-' .. ARGV[3])
redis.call('PEXPIRE', key, window_ms)
return 1
"""

r = redis.Redis(host="redis.internal", port=6379)
sliding_window = r.register_script(SLIDING_WINDOW)

def allow(tenant_id: str, endpoint: str, limit: int, window_ms: int) -> bool:
    # The hash tag keeps all of a tenant's keys in one cluster slot.
    key = f"rl:{{tenant:{tenant_id}}}:{endpoint}"
    request_id = uuid.uuid4().hex
    return sliding_window(keys=[key], args=[limit, window_ms, request_id]) == 1

if allow("42", "POST:/v1/orders", limit=100, window_ms=60_000):
    print("handle request")
else:
    print("reject with 429")

A Lua script for a sliding window log, run through redis-py, that reads the time with TIME and prunes, counts and adds entries atomically, so replicas cannot race between the check and the write.

Memory and latency compared: an illustrative estimate

Illustrative estimate (assumed numbers, not a benchmark). Assume 50,000 active client keys, 10 rate-limited endpoints, a limit of 100 requests per minute, and an average of 20 requests sitting in each window. Assume about 100 bytes per stored entry, including key and structure overhead. Measure your own overhead before you size anything.

Tracked pairs = keys × endpoints = 50,000 × 10 = 500,000.

• Sliding window log: pairs × entries in window × bytes per entry = 500,000 × 20 × 100 ≈ 1 GB. If every key sits at its limit, this grows to about 5 GB.
• Sliding window counter: pairs × 2 windows × bytes per counter = 500,000 × 2 × 100 ≈ 100 MB, whatever the traffic level.

The log stores one entry per request, so its memory grows with traffic. The counter stores fixed-size state per key, so its memory stays flat. In-process, either figure is also multiplied by the number of replicas, because each replica holds its own copy.

The limiter check sits on the request path, so a slow Redis call makes the whole request slow. If a request runs two sequential checks, for example a global limit and a per-endpoint limit, about 1 − 0.99² ≈ 2% of requests will hit at least one p99 lookup.

What this means for cost: the counter algorithm fits on a small Redis instance, so the infrastructure cost is modest. Most of the extra cost is engineering time: writing atomic scripts, working within hash-slot constraints, and deciding how to behave when Redis is slow or unavailable.

When Redis is slow or down: fail open, fail closed, or fall back

Every limiter call is a network hop on the request path, so give it a strict timeout. Set that timeout well below the endpoint's latency budget, typically a few milliseconds rather than the client library's default. Pair it with a circuit breaker so a struggling Redis isn't hit with retries. A slow Redis is worse than a dead one: without a timeout, every request waits on it, and the limiter becomes the outage.

Once the timeout fires, the code has to do something, and that choice should be made deliberately per endpoint.

With local fallback, each replica enforces the shared limit divided by the replica count. The limit then drifts whenever the replica count changes or a load balancer distributes traffic unevenly.

Two hybrids reduce how much you depend on Redis in the first place:

• A coarse local limiter in front of Redis. Set it at several times the per-replica share. It rejects obvious floods without a network call, so abusive bursts never reach Redis, and it keeps working during an outage.
• Local token batches. Each replica claims a batch of tokens from Redis, spends them locally, and reconciles periodically. This cuts round trips. The cost is that up to one batch per replica can be overspent.

Approximate shared limits are acceptable when the limit protects capacity rather than enforcing a contract. Overshooting by a batch matters little for load shedding. It does matter for billed quotas or brute-force protection.

Copyimport time
import redis

r = redis.Redis(host="redis.internal", port=6379,
                socket_timeout=0.005, socket_connect_timeout=0.005)

REPLICAS = 4  # local share drifts if this is stale or traffic is uneven

FAIL_MODE = {"/login": "closed", "/search": "open", "/orders": "local"}

local_counts = {}  # per-replica fallback counters, keyed like Redis

def allow(endpoint, client_id, limit, window=60):
    bucket = int(time.time() // window)
    key = f"rl:{endpoint}:{client_id}:{bucket}"
    try:
        pipe = r.pipeline()
        pipe.incr(key)
        pipe.expire(key, window)
        count, _ = pipe.execute()
        return count <= limit
    except redis.exceptions.RedisError:  # includes TimeoutError, ConnectionError
        mode = FAIL_MODE.get(endpoint, "local")
        if mode == "open":
            return True   # let traffic through; overload is survivable
        if mode == "closed":
            return False  # caller returns 429/503; protects login, payments
        # Local fallback: enforce this replica's share of the shared limit.
        if len(local_counts) > 100_000:
            local_counts.clear()  # crude bound on memory during long outages
        local_counts[key] = local_counts.get(key, 0) + 1
        return local_counts[key] <= max(1, limit // REPLICAS)

A fixed-window Redis limiter with millisecond timeouts and an explicit per-endpoint choice to fail open, fail closed, or fall back to a per-replica local share when Redis errors or times out.

Alternatives to per-endpoint rate limiting

Request-rate limits are one way to protect a route. Depending on what you are protecting, one of these may fit better or work alongside them:

• Edge or gateway rate limiting. Limits enforced at a reverse proxy, load balancer or API gateway reject traffic before it reaches the monolith. They are better for volumetric abuse and keep the application simple. They are often coarser, though, because the proxy may not see the matched route template or the authenticated identity.
• A single global or per-client limit. One budget per client across all routes is simpler to operate and explain. It is enough when routes have similar costs.
• Concurrency limits. Cap the number of in-flight requests per route, for example with a semaphore around the export handler, instead of the number per minute. This tracks actual resource use more directly for slow, expensive endpoints, because ten concurrent ten-second exports cost more than ten quick ones spread across a minute.
• Adaptive load shedding. Reject requests when the server itself is overloaded, based on signals such as queue depth, latency or connection-pool saturation, rather than on fixed per-client budgets. This protects capacity but does not provide fairness between clients or enforce contracts.
• Queues and asynchronous jobs. Move expensive work such as reports and bulk imports to a background job queue with bounded worker counts. Clients get a job ID instead of a blocked request, and throughput is controlled by the number of workers.
• Caching. For expensive but repeatable reads, caching responses can remove the load that a rate limit would otherwise ration.
• Usage quotas and billing-level metering. For contractual limits measured over days or months, a quota system that records usage durably may be more appropriate than a short-window limiter.

These are not mutually exclusive. A common layering is edge limits for floods, per-endpoint limits for fairness and expensive routes, and concurrency limits or load shedding as a last line of defense for the process itself.

Response semantics, testing and the impact on engineering time

Response semantics

Reject with 429 Too Many Requests and a Retry-After header that gives the number of seconds until the key's budget refills. Well-behaved clients use it to back off instead of retrying in a tight loop. Remaining-quota headers such as X-RateLimit-Remaining are an optional convention, not a universal standard. Add them only if clients will actually pace themselves with them, and document their meaning.

Use one clock source per limiter. In-process limiters should read a monotonic clock so wall-clock adjustments cannot reset windows. Redis-backed limiters should compute time inside the script using Redis's own TIME. Otherwise replicas with skewed clocks will disagree about where a window starts.

Testing and observability

Inject the clock so unit tests can advance time deterministically and assert exact boundary behavior without sleep. For Redis, run a load test that sends traffic through several replicas at once and confirm that the combined accepted count matches one shared budget, not one budget per replica. In production, emit allowed and rejected counts per endpoint, along with limiter latency, so you can tell a tight limit apart from a slow limiter.

Decision checklist

• Is there a single replica, or is a per-replica approximate limit acceptable? Stay in-process.
• Must replicas share an exact budget, for example for billed quotas or contractual limits? Move to Redis.

Starting in-process keeps the network, failure modes and extra infrastructure out of the system until a real requirement calls for them. Moving to Redis adds operational cost:

• A dependency to provision and monitor
• A round trip on every request
• A fail-open or fail-closed policy that must be tested

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to monolith rate limiting per endpoint, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.