Rate Limiting a Monolith Without Redis: In-Memory vs Postgres

What a rate limit means depends on where its state lives

Teams add rate limits for a few recurring reasons. They want to slow down credential stuffing and scraping, stop one tenant from crowding out the others, and protect endpoints that are expensive to serve, such as report generation, search, or anything that calls a paid third-party API. The policy is usually written as a single sentence: "100 requests per minute per API key." That sentence only becomes a specification once you decide where the counter lives and which requests can see it.

The counter's scope is the limit's scope

A counter held in process memory is shared only by the requests that process handles. Suppose the monolith runs four worker processes on each of three replicas, with a load balancer spreading traffic evenly. You now have twelve independent counters. A client can make roughly 1,200 requests per minute before every counter trips, and the exact figure depends on how requests happen to be distributed. Threads inside a single process do share memory, but only if the counter is protected against concurrent updates. A Postgres row is different: every worker on every replica reads and writes the same state, so "100 per minute" means 100.

Who should consider skipping Redis

Redis is the default answer because it gives you shared state with fast atomic increments. It is also another stateful service to provision, secure, monitor, fail over, and upgrade. Running it may not be worth it if you are a monolith team that already operates Postgres and has modest limiter traffic, or a team that can accept per-process limits.

Copy      clients sharing one API key (policy: 100 requests/min per key)
                                     |
                                     v
                            +-----------------+
                            |  load balancer  |
                            +-----------------+
                                     |
            +------------------------+------------------------+
            v                        v                        v
+----------------------+ +----------------------+ +----------------------+
| replica A            | | replica B            | | replica C            |
| +--+ +--+ +--+ +--+  | | +--+ +--+ +--+ +--+  | | +--+ +--+ +--+ +--+  |
| |w1| |w2| |w3| |w4|  | | |w1| |w2| |w3| |w4|  | | |w1| |w2| |w3| |w4|  |
| +--+ +--+ +--+ +--+  | | +--+ +--+ +--+ +--+  | | +--+ +--+ +--+ +--+  |
| 4 in-memory counters | | 4 in-memory counters | | 4 in-memory counters |
+----------------------+ +----------------------+ +----------------------+
            |                        |                        |
            +------------------------+------------------------+
                                     |  Postgres mode: every worker on
                                     v  every replica hits the same row
                     +-------------------------------+
                     | Postgres-backed limiter       |
                     | one row per (key, window)     |
                     | shared by all 12: 100/min     |
                     +-------------------------------+

  In-memory: 12 separate counters, so roughly 12 x 100 = ~1,200/min per key
  Postgres:  1 shared counter, so 100/min means 100/min

With in-memory limiters, each of the 12 worker processes keeps its own counter, so a 100/min policy allows roughly 1,200/min in total. A Postgres-backed limiter gives every worker on every replica one shared row, so the limit holds at 100/min.

How the common algorithms map to memory and to Postgres

Every algorithm below answers the same question: how many requests has this key made recently? They differ in how much history they keep, and that history becomes rows, writes and dead tuples once it lives in Postgres.

Fixed window keeps one counter per key per window, such as "requests in the minute starting 12:00". It is cheap in both places: a map entry in memory, or one row incremented by a single UPDATE. The weakness is the boundary. A client can send the full limit at 12:00:59 and the full limit again at 12:01:00, so it gets roughly 2x the limit in about one second.

Sliding window log stores a timestamp for every accepted request and counts the ones inside the window. It is exact, but its state grows with the limit. In memory that means N timestamps per key. In Postgres it means one row per request: an insert, a range count and a delete of expired rows on every call. The deletes leave dead tuples that vacuum must reclaim, and the index churns constantly.

Sliding window counter keeps the current and previous fixed-window counts and weights the previous one by how much of it still overlaps the window. That gives a close approximation with two integers per key.

Token bucket stores a token count and a last-refill timestamp, and refills lazily on each access. In Postgres, a single atomic UPDATE can do the refill and the deduction together.

Building a correct in-memory rate limiter

Each bucket needs only two fields: the current token count and the time of the last refill. Store them in a map keyed by client identity, such as an API key, user ID or normalized IP. Guard each bucket with a lock, or shard the map across a fixed number of lock-protected segments so unrelated clients don't contend on one mutex. The refill, the check and the decrement must happen inside the same critical section. If you read the count, release the lock and then decrement, two concurrent requests can both see one remaining token and both pass.

Refill lazily. On each request, compute elapsed = now - last_refill, add elapsed * rate tokens capped at capacity, set last_refill = now, and then try to take a token. This removes background timers, which would otherwise cost CPU per key whether or not the client is active. Read time from a monotonic clock. A wall clock can jump backward after an NTP correction, and a negative elapsed time would drain tokens.

Memory needs a hard bound, because every distinct key creates an entry and spoofed headers or rotating IPs can create keys without limit. Evicting idle keys loses nothing when done correctly. A bucket that has been idle long enough to refill to capacity behaves the same as a missing bucket, since a new one also starts full. Sweep periodically for entries older than capacity / rate. Also enforce a maximum entry count with LRU eviction. Under a flood of unique keys, the LRU cap is the guarantee that keeps memory bounded, and the sweep only handles normal idle cleanup.

Copyimport threading
import time
from collections import OrderedDict

class TokenBucketLimiter:
    def __init__(self, rate=5.0, capacity=10, max_keys=100_000):
        self.rate = rate                    # tokens added per second
        self.capacity = capacity
        self.max_keys = max_keys            # hard memory bound
        self.idle_ttl = capacity / rate     # idle this long == full bucket
        self.buckets = OrderedDict()        # key -> [tokens, last_refill], LRU order
        self.lock = threading.Lock()        # one lock for brevity; shard it in production

    def allow(self, key):
        with self.lock:  # refill, check and decrement in ONE critical section
            now = time.monotonic()  # read under the lock so time never goes backward
            b = self.buckets.get(key)
            if b is None:
                b = self.buckets[key] = [self.capacity, now]  # new bucket starts full
                if len(self.buckets) > self.max_keys:
                    self.buckets.popitem(last=False)  # evict least recently used
            else:
                self.buckets.move_to_end(key)
                b[0] = min(self.capacity, b[0] + (now - b[1]) * self.rate)  # lazy refill
                b[1] = now
            if b[0] >= 1:
                b[0] -= 1
                return True
            return False

    def sweep(self):
        # Call periodically. LRU order matches last_refill order, so stop at the first active key.
        with self.lock:
            now = time.monotonic()
            while self.buckets:
                key, b = next(iter(self.buckets.items()))
                if now - b[1] < self.idle_ttl:
                    break
                del self.buckets[key]

A thread-safe token bucket limiter in Python that refills lazily from a monotonic clock, does the refill, check and decrement under one lock, and bounds memory with an LRU cap plus a sweep of keys idle long enough to be full.

When single-node throttling breaks, and when to avoid it

Each copy of that bounded map belongs to one process, and that is where the in-memory limiter stops being a limit.

Every process gets its own allowance

Pre-fork application servers run several worker processes, and each one builds its own limiter after the fork. A pool of threads inside one process can share a single locked map. The moment the server spawns separate processes, though, each holds an independent copy, whether those processes run threads or event loops. Horizontal replicas behind a load balancer multiply this again. With a limit of N per process across K processes, a client can be admitted up to about N × K times. As an illustrative estimate, take a limit of 100 per minute, 4 workers per host and 3 hosts. That gives 12 processes and up to 1,200 requests per minute.

Distribution makes it uneven

The effective limit is not even a stable N × K. Round-robin balancing spreads one client across every process, so the client approaches the full multiplied allowance. Sticky sessions or connection reuse pin other clients to a single process, and those clients hit the real limit of N. Two clients with identical traffic get different treatment depending on routing they cannot see.

Restarts hand out fresh budgets

State lives only in process memory, so every restart, crash, autoscaling event or deploy resets the counters. Rolling deploys make this worse. Old and new instances overlap, which briefly raises K, and every new instance starts with empty counters.

Use in-memory limiting only while you genuinely run one process, or when over-admission by a factor of K is acceptable.

Implementing a Postgres rate limit with atomic statements

Postgres removes the race by doing the read and the write in one statement. A fixed-window limiter needs a table keyed on (key, window_start) with an integer count, where that pair is the primary key. Each request runs one INSERT ... ON CONFLICT (key, window_start) DO UPDATE SET count = <table>.count + 1 RETURNING count. It sets window_start with date_trunc('minute', now()) or a similar bucketing expression. The first request in a window inserts 1. Later requests take the conflict path and increment the counter in place. The application gets the new count in the same round trip and compares it to the limit. It never runs a SELECT first, so no gap exists in which two requests can read the same stale value.

A token bucket stores tokens and last_refill per key. A single UPDATE ... RETURNING does the refill and the spend in one step:

• Refill: set tokens to the lesser of capacity and the current tokens plus elapsed seconds since last_refill, multiplied by the refill rate, then subtract one.
• Spend: set last_refill = now().
• Reject: add a WHERE clause that requires the refilled balance to be at least one. If no row comes back, the request is rejected.

Both designs read time from now() and not from the application. Every app node then uses one clock, and skew between hosts cannot widen or shrink a window.

Correctness on a hot key comes from row-level locking. A second transaction that updates the same row waits until the first one commits. Under the default READ COMMITTED isolation, the second transaction then re-evaluates its expressions against the newly committed row version, so increments are never lost. This also means each key is processed serially. Run these statements in autocommit or in very short transactions, because any work done while the row lock is held adds directly to the queue behind it.

Copy-- Fixed window: one row per key per minute.
CREATE TABLE rate_window (
  key          text        NOT NULL,
  window_start timestamptz NOT NULL,
  count        integer     NOT NULL,
  PRIMARY KEY (key, window_start)
);

-- Increment and read in one statement. The DB clock picks the bucket.
INSERT INTO rate_window AS rw (key, window_start, count)
VALUES ($1, date_trunc('minute', now()), 1)
ON CONFLICT (key, window_start)
DO UPDATE SET count = rw.count + 1   -- waits on the row lock if another request holds it
RETURNING count;                     -- app compares this to the limit, e.g. 100

-- Token bucket: capacity 10, refill 2 tokens/second.
CREATE TABLE token_bucket (
  key         text             PRIMARY KEY,
  tokens      double precision NOT NULL,
  last_refill timestamptz      NOT NULL
);

-- Seed a full bucket on first use. This is a no-op if the row exists.
INSERT INTO token_bucket (key, tokens, last_refill)
VALUES ($1, 10, now())
ON CONFLICT (key) DO NOTHING;

-- Refill, check and spend in one UPDATE. No row returned means the request is rejected.
UPDATE token_bucket
SET tokens = LEAST(10, tokens + EXTRACT(EPOCH FROM now() - last_refill) * 2) - 1,
    last_refill = now()
WHERE key = $1
  AND LEAST(10, tokens + EXTRACT(EPOCH FROM now() - last_refill) * 2) >= 1
RETURNING tokens;
-- Run in autocommit. now() is the transaction start time, and a long transaction
-- would hold the row lock and stall every other request on this key.

A fixed-window counter uses a single INSERT ... ON CONFLICT DO UPDATE ... RETURNING. A token bucket refills, checks and spends in one UPDATE ... RETURNING. Both statements use the database clock and depend on row-level locks to serialize requests on the same key.

Operating the database limiter: cost, failure modes and keys

Keeping the table healthy

Give the table a primary key on the client key, or on the client key plus window start for fixed windows. Without one, every upsert does a scan, and ON CONFLICT has no unique constraint to target. Expired windows pile up, so delete them on a schedule. With time-based partitioning you can drop old partitions instead, which avoids the dead rows a large DELETE leaves behind.

Counter data is disposable, so an unlogged table is a reasonable fit. It skips WAL, which makes writes cheaper. In exchange, PostgreSQL truncates it after a crash and does not replicate it to standbys. After a failover, every client starts with fresh limits.

Each increment writes a new row version, so hot keys create dead tuples quickly. Tune autovacuum per table so it runs more often. Set a lower fillfactor and keep the updated columns out of indexes. Both let PostgreSQL use HOT updates, which avoid index bloat.

Cost on the hot path

Every limited request now does a database write and holds a pool connection while it runs. Under a traffic spike, which is exactly when the limiter matters, these writes compete with real work for the same connections. Decide in advance what happens if the check times out or errors:

• Fail open keeps the service available but leaves it unprotected.
• Fail closed keeps it protected but turns a database problem into a full outage.

Set a short statement timeout either way.

Choosing the key

Prefer a user ID or API key. Fall back to IP only for anonymous traffic. Read X-Forwarded-For only when the request came from a proxy you control, because clients can set that header to anything. Run the check after authentication, so you know who the caller is, and before expensive work, so rejected requests cost little. When you reject a request, return HTTP 429 with a Retry-After header set to the number of seconds until the window resets.

Copy-- Disposable counters: UNLOGGED skips WAL, but the table is truncated after a crash
-- and is not replicated to standbys, so limits reset on failover.
CREATE UNLOGGED TABLE rate_limit_counters (
    client_key   text        NOT NULL,  -- e.g. 'user:42' or 'apikey:ak_live_example'
    window_start timestamptz NOT NULL,  -- start of the fixed one-minute window
    hits         integer     NOT NULL DEFAULT 0,
    -- Lets ON CONFLICT find the row through the index instead of a table scan.
    PRIMARY KEY (client_key, window_start)
) WITH (
    fillfactor = 70,                       -- leave free space on each page for HOT updates
    autovacuum_vacuum_scale_factor = 0.0,  -- don't wait for a percentage of the table to be dead
    autovacuum_vacuum_threshold = 1000     -- vacuum after roughly 1000 dead tuples
);
-- Do not index hits. Updates to non-indexed columns can be HOT and avoid index bloat.

-- Hot path: one upsert per request, bounded by a short timeout.
BEGIN;
SET LOCAL statement_timeout = '50ms';  -- on timeout, the app applies its fail-open or fail-closed policy
INSERT INTO rate_limit_counters (client_key, window_start, hits)
VALUES ('user:42', date_trunc('minute', now()), 1)
ON CONFLICT (client_key, window_start)
DO UPDATE SET hits = rate_limit_counters.hits + 1
RETURNING hits;  -- app compares this to the limit and returns 429 + Retry-After if exceeded
COMMIT;

-- Scheduled cleanup of expired windows. Run often so each DELETE stays small.
DELETE FROM rate_limit_counters
WHERE window_start < now() - interval '5 minutes';

An unlogged fixed-window counter table with a composite primary key, fillfactor and per-table autovacuum settings, plus the timeout-bounded upsert used on the hot path and a scheduled cleanup of expired windows.

Choosing an approach: alternatives, hybrid design, and engineering cost

Each option spends a different resource: accuracy, latency, or operational effort.

In the hybrid design, a per-node in-memory bucket with generous limits rejects obvious abuse without touching the database. Postgres stays the authoritative per-account counter, so a flood never turns into a flood of writes.

Illustrative estimate (assumed numbers, not a benchmark): extra writes/s = R × (1 − f), where R is limited requests per second and f is the fraction the pre-filter rejects. With R = 500 and f = 0.2, that is 500 × 0.8 = 400 single-row upserts per second.

Staying on Postgres avoids running another stateful service. You add no new failover, backups, credentials or alerts. A dedicated store becomes worth that cost in three cases:

• Limiter writes compete with transactional traffic.
• Hot keys cause row-lock waits.
• The extra round trip breaks your latency budget.

How to choose by topology:

• One node: use the in-memory limiter.
• A few nodes that can tolerate N× over-admission: use in-memory buckets with each node's limit set to the global limit divided by N.
• Exact per-account quotas: use Postgres or the hybrid design.
• Many nodes at high request rates: use a dedicated store.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to rate limiting monolith without redis, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.