What is per tenant rate limiting in a monolith, and why is it used?

Per tenant rate limiting caps how much of a shared application's capacity any single customer can consume in a given period. In a monolith the goal is not only to reject excess traffic at the edge. The real job is to protect the internal resources every tenant draws from: the database connection pool, worker threads, background job queues and outbound API quotas. One tenant's bulk export can hold most of the pool's connections while everyone else's lightweight reads queue behind it.

This is the noisy neighbor problem. In a multi-tenant monolith, tenants share one process and one pool, so there is no natural boundary that stops one heavy workload from degrading latency for all the others. Fair usage limits create that boundary in software.

Who should adopt it

It pays off for SaaS backends on shared infrastructure where tenant load is uneven, for example a few large accounts alongside many small ones. It is usually not worth the effort when:

  • you run single-tenant deployments
  • you have only a handful of tenants you can manage by hand
  • tenants are already isolated on separate databases or clusters

Identify the tenant early

Resolve the tenant ID before any expensive work happens. The usual sources are a claim in the auth token, a lookup from the API key, or the request subdomain. Keying by IP fails because many users from one customer share a NAT or proxy, while one tenant's integrations can spread across many addresses. Keying by user fails because a tenant with 500 users gets 500 times the budget. Fairness has to be measured at the tenant level.

When should it be used?

Use per tenant limits when your operational history shows that one tenant can degrade service for the others. These signals are concrete:

  • Latency incidents trace back to one account. p95 or p99 latency rises for everyone, and the cause turns out to be a single tenant's import, export, report or misbehaving integration.
  • Connection pool or worker exhaustion. Pool checkout timeouts or saturated worker threads appear while overall request volume looks normal. This means a few tenants are holding resources for a long time.
  • Background job delays. Time-sensitive jobs such as password resets, notifications and webhooks sit behind one tenant's bulk work in a shared queue.
  • Shared outbound quotas. Your app calls a third-party API with one account-wide quota, so one tenant's usage can exhaust it for everyone.
  • You sell tiered plans. If plans promise different throughput, you need enforcement that matches what you sell.
  • You expose a public API or allow customer-written integrations. Client code you don't control will eventually loop, retry without backoff or poll aggressively.

Timing matters too. Add limits before onboarding a customer whose traffic is much larger than your current largest tenant, not after their first sync job takes the database down. Start in shadow mode (described below) so you learn real usage before you enforce anything.

If none of these signals apply and the conditions in "Who should adopt it" point the other way, a simple edge-level request limit is usually enough.

How does it work?

At a high level, per tenant rate limiting has four parts:

  1. Attribute every unit of work to a tenant. Middleware tags each request, and each job enqueued on its behalf, with a tenant ID.
  2. Keep a budget per tenant. Each tenant, and optionally each route class, has a counter or token bucket held in shared state. Its size comes from the tenant's plan plus any override.
  3. Charge the budget before doing work. Each request or job deducts a cost, either a flat 1 or a weight that reflects how expensive it is. If the budget can't cover the cost, the work is rejected with a 429 or deferred.
  4. Cap what a tenant holds, not just what it starts. Per-tenant concurrency limits on DB connections, job workers and expensive features stop a tenant that is within its request rate from monopolizing long-lived resources.

The sections below go through each part in detail: where the checks sit in the request lifecycle, which algorithm to use for the budget, how to keep the state consistent across instances, and how to size and tune the limits.

How does per tenant rate limiting work across the request lifecycle?

A request passes through several points where tenants compete for capacity, and each point needs its own control.

Tenant resolution. Middleware runs first and derives a tenant ID from the API key, a token claim or the subdomain, before any expensive work happens. Every later limit is keyed on this value. Requests that cannot be resolved should go into a separate, strict bucket rather than a shared default that any tenant can exhaust.

Request-layer limiter. A rate check then admits the request or rejects it with a 429. It is cheap and caps arrival rate, but it only counts how many requests start.

Database connection pool. Request rate does not measure load. Suppose a tenant stays well inside 10 requests per second, but each request runs a 30-second report query. That tenant can still hold every connection in the pool. Cap concurrent connections per tenant, for example with a per-tenant semaphore around checkout. Also set statement timeouts so one query cannot pin a connection indefinitely.

Background jobs. With a single FIFO queue, one tenant's large import can delay every other tenant's password-reset emails. Partition queued work by tenant and have workers pull round-robin or weighted across tenants. Cap in-flight jobs per tenant as well.

Bulkheads. Give expensive features such as exports, full-text search and PDF rendering their own worker pools or concurrency limits. When one of them saturates, the problem stays inside that feature.

Rate limits control how often work starts. Concurrency caps and timeouts control how much capacity a tenant holds at once. You need both.

          incoming request: API key / token claim / subdomain
                                    |
                                    v
+------------------------------------------------------------------------+
| 1. Tenant resolution middleware -> tenant_id (unresolved -> strict)    |
+------------------------------------------------------------------------+
                                    |
                                    v
+------------------------------------------------------------------------+
| 2. Request-layer limiter: per-tenant rate, caps starts ---> 429        |
+------------------------------------------------------------------------+
           |                        |                         |
           v                        v                         v
+---------------------+  +----------------------+  +---------------------+
| 3. DB conn pool     |  | 4. Job queues        |  | 5. Bulkheads        |
| per-tenant          |  | partitioned by       |  | separate pools for  |
| semaphore around    |  | tenant; round-robin  |  | exports, search,    |
| checkout            |  | or weighted pulls    |  | PDF rendering       |
| + statement         |  | + in-flight cap      |  | saturation stays    |
| timeout             |  | per tenant           |  | inside feature      |
+---------------------+  +----------------------+  +---------------------+
           |                        |                         |
           v                        v                         v
+------------------------------------------------------------------------+
|  Shared capacity: DB connections, worker threads, CPU, memory          |
+------------------------------------------------------------------------+
  Rate limit = how often work starts. Concurrency caps + timeouts =
  how much capacity a tenant holds at once.

A request is tagged with a tenant ID and checked against a per-tenant rate limit, and per-tenant concurrency caps, timeouts, fair job queues and feature bulkheads then limit how much shared capacity any one tenant can hold.

Which rate limiting algorithm should you choose: fixed window, sliding window, token bucket or leaky bucket?

Within the rate limit layer, the algorithm determines how much burst a tenant can produce and how much state you store for each tenant.

Algorithm Burst tolerance Smoothness State per tenant Complexity
Fixed window Uncontrolled at boundaries Poor One counter Lowest
Sliding window log None beyond limit High One timestamp per request Moderate
Sliding window counter Small, approximated Good Two counters Low
Token bucket Configurable (bucket size) Good on average Token count + last refill time Low
Leaky bucket (queue) Absorbed by queue, not passed through Highest Queue or next-allowed time Moderate

A fixed window resets its counter at each boundary. Take a limit of 100 per minute. A tenant can send 100 requests at 12:00:59 and another 100 at 12:01:00, which puts 200 requests through in about one second. Clock skew makes this worse. If each instance of the monolith derives window boundaries from its own clock, the windows don't line up, and a tenant whose requests are spread across instances can land in a fresh window on each one.

A token bucket has two parameters with separate meanings. The refill rate sets the sustained average. The bucket size sets the largest burst allowed. This fits tenant traffic well, because real clients batch work: a page load fans out, or a sync job runs on the hour. You can absorb a reasonable spike and still cap long-run consumption. Its state is two numbers, and you can refill lazily on each request using elapsed time, so no background timer is needed.

Choose a leaky bucket only when the downstream needs a strictly even rate. Choose a sliding window log only when exact counts matter more than memory.

How do you implement per tenant limits with shared state and cost weighting?

Where the counters live

In-process counters are fast, but each instance enforces its own copy of the limit. With N replicas behind a load balancer, a tenant can get up to N times its quota. Use them only for a single instance, or as a coarse local guard in front of a shared limit. For accurate per tenant rate limiting, keep bucket state in a shared in-memory store that every instance reads and writes.

Avoiding races

A separate read, then decide, then write lets two concurrent requests both see the last token and both succeed. Make the check atomic. For fixed windows, an atomic increment with an expiry is enough. For a token bucket, run a server-side script that refills from elapsed time, compares the result to the requested cost, decrements, and writes back in one step. The code example below does this.

Key buckets by tenant and limit type, such as rl:{tenant_id}:api_read and rl:{tenant_id}:export. This keeps a burst of exports from using up a tenant's interactive budget.

Cost weighting

Counting every request as 1 lets a cheap health check and a full-table report draw from the same budget. That is how one noisy neighbor slows everyone else down. Instead, charge a cost by endpoint class, or after the fact by rows scanned or job duration. Static estimates go stale, and clients learn which parameters are underpriced. Compare charged cost against measured cost and re-tune the weights regularly.

When the store is down

Fail open, with a local fallback limit, for paths that serve users interactively. Fail closed for expensive or abusable operations, because skipping the limit on those can overload the shared database.

import redis

TOKEN_BUCKET = """
local capacity = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])   -- tokens refilled per second
local cost = tonumber(ARGV[3])   -- weight of this request
local t = redis.call('TIME')     -- server clock, so app-host clock skew does not matter
local now = tonumber(t[1]) * 1000 + math.floor(tonumber(t[2]) / 1000)
local b = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(b[1]) or capacity
local ts = tonumber(b[2]) or now
tokens = math.min(capacity, tokens + (now - ts) * rate / 1000)
local allowed = 0
if tokens >= cost then
  tokens = tokens - cost
  allowed = 1
end
redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
-- Expire idle buckets once they would be full again anyway
redis.call('PEXPIRE', KEYS[1], math.ceil(capacity / rate * 1000))
return allowed
"""

r = redis.Redis(host="ratelimit-redis.internal", port=6379, socket_timeout=0.05)
bucket = r.register_script(TOKEN_BUCKET)

COSTS = {"health": 1, "list": 5, "report": 50}

def allow(tenant_id, limit_type, endpoint_class, capacity, rate, fail_open):
    key = f"rl:{tenant_id}:{limit_type}"  # separate budget per limit type
    cost = COSTS[endpoint_class]
    try:
        return bucket(keys=[key], args=[capacity, rate, cost]) == 1
    except redis.exceptions.RedisError:
        # Store unavailable: interactive paths fail open (pair with a local
        # per-instance fallback limit); expensive paths fail closed.
        return fail_open

An atomic Redis token bucket, run as a Lua script, that charges each request a weight based on its endpoint class, uses a separate key for each tenant and limit type, and makes an explicit fail-open or fail-closed choice when the store is unreachable.

Sizing a token bucket from DB capacity: an illustrative estimate

Refill rate should come from the resource that fails first, usually the database, not from a number that sounds reasonable. Here a token equals one query, so cost weighting from the previous section carries straight through.

Illustrative estimate (assumed numbers, not a benchmark)

  • Assumed: connection pool of 50, average query time 10 ms, 30% headroom.
  • Assumed tier mix: 400 Free (weight 1), 80 Pro (weight 5), 20 Enterprise (weight 20).

Formulas:

  • budget = pool_size × (1000 / avg_query_ms) × (1 − headroom) = 50 × 100 × 0.7 = 3,500 tokens/s
  • total_weight = Σ(tenants_in_tier × weight) = 400 + 400 + 400 = 1,200
  • refill_per_tenant = budget × weight / total_weight
  • capacity = refill × burst_seconds (assumed 10 s)
Tier Refill (tokens/s) Capacity (10 s burst) Tier share of budget
Free ~2.9 ~29 1/3
Pro ~14.6 ~146 1/3
Enterprise ~58 ~583 1/3

Capacity may sum to more than the database can absorb at once. That is deliberate: it rests on the assumption that tenants rarely burst together. Global admission control has to cover the case where they do.

Strict limits reserve capacity for every tenant, including idle ones. If only 50 tenants are active at 3 a.m., most of the 3,500 tokens/s goes unused while an active Pro tenant gets throttled at 14.6. Borrowing schemes and work-conserving schemes recover that idle capacity. Both make the limit a tenant gets depend on what other tenants are doing, so you can't promise a customer a fixed number. A common compromise is to publish the strict floor as the contract and treat borrowed capacity as best-effort that can be taken away at any time.

How should tiered limits, overrides and 429 responses be handled?

Derive limits from plan, then layer overrides

Resolve each tenant's limit in two steps. Start with a default from their plan tier, then apply any per-tenant override on top. Keep overrides in a database table or config store that the limiter reads and caches for a short TTL, such as 30–60 seconds. Ops can then raise a limit during an incident or a customer's launch without a deploy, and the change takes effect fleet-wide within one cache cycle.

An override record should carry more than a number. A good record looks something like tenant_id: acme, route_class: bulk_export, rate: 200/s, burst: 1000, reason: "nightly sync peak", expires_at: 2026-12-31. The reason and expiry matter because overrides outlive the people who granted them. Large tenants whose normal peaks exceed the tier limit should get a permanent, documented override rather than a string of temporary bumps. Otherwise their everyday traffic keeps tripping alerts.

Make 429s actionable

A rejected request should tell the client exactly when to try again. A typical response is 429 Too Many Requests with Retry-After: 12, along with remaining-quota headers such as X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset. A body naming the limit that was hit, such as "limit": "bulk_export", saves a support ticket.

Publish the limits per tier and route class, and state whether Reset is in seconds or a timestamp. Tell clients to honor Retry-After and add jitter. Without jitter, every throttled client retries at the same instant and hits the limit again.

import random
from dataclasses import dataclass
from datetime import date

@dataclass(frozen=True)
class Limit:
    rate: int   # sustained requests per second
    burst: int  # token bucket capacity

PLAN_LIMITS = {
    ("free", "bulk_export"): Limit(rate=5, burst=20),
    ("pro", "bulk_export"): Limit(rate=50, burst=200),
}

OVERRIDES = {
    ("acme", "bulk_export"): {"limit": Limit(rate=200, burst=1000),
                              "reason": "nightly sync peak", "expires_at": date(2026, 12, 31)},
}

def resolve_limit(tenant_id, plan, route_class, today=None):
    today = today or date.today()
    override = OVERRIDES.get((tenant_id, route_class))
    if override and override["expires_at"] >= today:
        return override["limit"]  # an unexpired override wins over the plan default
    return PLAN_LIMITS[(plan, route_class)]

def too_many_requests(limit, route_class, retry_after_s, reset_epoch):
    headers = {
        "Retry-After": str(retry_after_s),       # delay in seconds
        "X-RateLimit-Limit": str(limit.rate),
        "X-RateLimit-Remaining": "0",
        "X-RateLimit-Reset": str(reset_epoch),   # documented to clients as a Unix timestamp
    }
    body = {"error": "rate_limited", "limit": route_class, "retry_after": retry_after_s}
    return 429, headers, body

def backoff_seconds(retry_after_s):
    return retry_after_s + random.uniform(0, retry_after_s * 0.5)

Resolves a tenant's limit from its plan tier plus an expiring override record, builds a 429 response with Retry-After, remaining-quota headers and a body naming the limit hit, and shows a client backoff that adds jitter to Retry-After.

How do you observe and roll out limits, and what are the alternatives?

What to measure

Emit three signals per tenant: requests allowed versus throttled (with route class and limit name), resource consumption (DB time, weighted cost units), and a top-consumers view. Throttle counts alone are misleading. A tenant can stay under its request limit while saturating the database with expensive queries, which is why cost-weighted usage matters.

Tenant ID as a metrics label works for hundreds of tenants but becomes expensive at tens of thousands, because every label combination is a separate time series. Keep low-cardinality metrics aggregated by tier, and send per-tenant detail to logs or a periodic top-N export. The limiter store grows the same way: one key per tenant per route class per window. Set TTLs so that idle tenants' keys expire instead of accumulating.

Rolling out

Run each new limit in shadow mode first. Evaluate it on every request, log when it would have rejected, and enforce nothing. Shadow logs show which tenants would be hit and whether those tenants are abusive or legitimate heavy users who need overrides. Enforce once false positives are understood.

Alternatives

Approach Strength Gap
Edge/gateway limiting Stops floods before app Can't see per-request cost
Per-tenant infra or cells Strong isolation Higher infra and ops cost
Queue-only backpressure Smooths bursts One tenant can still fill the queue
Autoscaling Absorbs real growth Shared DB still saturates

An in-house limiter costs ongoing maintenance: shared state, tuning, and override tooling. Weigh that against the engineering hours currently lost to noisy-neighbor incidents and to manually throttling individual tenants during an outage.

How can it save engineering time?

Most of the time savings come from replacing repeated manual, incident-driven work with a mechanism that runs on its own.

  • Fewer and shorter noisy-neighbor incidents. Without limits, a runaway tenant typically means someone is paged, digs through slow-query logs to find the tenant, and then improvises a fix: disabling a feature flag, killing queries or contacting the customer. With per-tenant concurrency caps and budgets, the heavy tenant gets throttled automatically and other tenants are unaffected. Often no one needs to be paged.
  • Faster diagnosis. Per-tenant throttle and cost-weighted usage metrics answer "who is causing this?" directly. Without them, engineers have to correlate request logs with database load by hand.
  • Limit changes without deploys. Overrides stored in a cached table let ops or support raise a limit for a customer launch or an incident in minutes. Without them, an engineer has to make a code change, get it reviewed and deploy it.
  • Fewer support escalations. A 429 with Retry-After and a body naming the limit that was hit lets client developers fix their own retry logic. Otherwise they open a ticket that ends up with an engineer.
  • Safer rollouts. Shadow mode shows who would be affected before enforcement starts. This avoids the cycle of enforcing a limit, breaking a legitimate integration, rolling back and starting again.
  • Postponing bigger re-architecture. Software isolation inside the monolith can delay the need to split tenants onto separate databases or cells. Those projects take far more engineering time to build and operate, so delaying one until it is truly necessary is a large saving.

Illustrative estimate (assumed inputs, not a measured result). Assume a team has 2 noisy-neighbor incidents per month, each involving 3 engineers for 4 hours of triage and mitigation, plus 5 hours per month spent on manual throttling and the related support escalations. That comes to 2 × 3 × 4 + 5 = 29 engineer-hours per month. If limits prevented or automatically contained most of those incidents, the recovered time would be compared against the build cost (the limiter, override table, metrics and shadow mode) and the ongoing tuning cost described in the previous section. Use your own incident history in place of these assumed numbers before deciding.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to per tenant rate limiting, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.