In-Process Cache vs Redis: Caching Layers for a Monolith

How an in-process LRU cache and Redis each work in a monolith

An in-process LRU cache is a bounded map inside the application's own heap. Each entry holds a live object reference. When the map reaches its size limit, the least recently used entry is evicted. A lookup is a hash-table read in local memory. There is no serialization, no socket, and no other process involved. The cache lives and dies with the process, so a restart or deploy empties it, and each replica keeps its own private copy.

Redis is a separate server process that the application reaches over the network. Every write turns an object into bytes, and every read sends a request, waits for the reply, and turns the bytes back into an object. In the code example below, the LRU read is a single map lookup. The Redis cache-aside read adds a client round trip, a deserialization step on a hit, and a serialization step when it writes back after a miss. In return, all replicas share one copy of the data, and that copy survives application restarts and deploys.

Teams adopt either option for the same reasons: to cut repeated database queries and to lower response latency. For a monolith, the speed gap between the two rarely decides anything. Both are usually far faster than the query they replace. The decision depends on consistency across replicas, memory pressure, and what happens on restart.

Copyimport json
from collections import OrderedDict

import redis


class LRUCache:
    """Bounded map in the app heap. Not thread-safe; add a lock if shared."""

    def __init__(self, max_entries=10_000):
        self.max_entries = max_entries
        self._data = OrderedDict()

    def get(self, key):
        if key not in self._data:
            return None
        self._data.move_to_end(key)  # mark as most recently used
        return self._data[key]       # live object reference, no copy or decode

    def put(self, key, value):
        self._data[key] = value
        self._data.move_to_end(key)
        if len(self._data) > self.max_entries:
            self._data.popitem(last=False)  # evict least recently used


r = redis.Redis(host="cache.internal", port=6379)  # placeholder host


def get_product(product_id, db):
    key = f"product:{product_id}"
    raw = r.get(key)                  # network round trip on every read
    if raw is not None:
        return json.loads(raw)        # deserialize bytes back into an object
    product = db.load_product(product_id)        # miss: query the database
    r.set(key, json.dumps(product), ex=300)       # serialize, second round trip, 5 min TTL
    return product

A minimal OrderedDict-based LRU that returns live objects from local memory, next to a redis-py cache-aside read that adds a network round trip and JSON serialization on each access.

Why replica count decides the question

Each replica of a monolith owns a private heap, so an in-process LRU cache becomes N independent caches. Each one warms on its own, holds its own copy of hot keys, and has to be told separately when data changes. An invalidation that reaches replica A but not replica B leaves B serving the old value until its TTL expires. A user whose requests are load-balanced across both can see a price or permission change appear, disappear, and reappear.

CopyIn-process LRU (private heaps)        Shared Redis
  LB ─┬─ R1 [cache] ─┐                  LB ─┬─ R1 ─┐
      ├─ R2 [cache] ─┼─ DB                  ├─ R2 ─┼─ Redis ─ DB
      └─ R3 [cache] ─┘                      └─ R3 ─┘
  3 copies, 3 misses per key            1 copy, 1 miss per key

Illustrative estimate (not a measurement)

Assumed inputs: 10,000 keys with uniform access, 1,000 req/s in total, round-robin load balancing, a 60 s TTL, and a model where each replica misses once per key per TTL window if it sees that key at all.

Skewed access narrows the gap, because very hot keys stay warm on every replica. Real figures depend on your workload, key distribution, and value size.

Copy  IN-PROCESS LRU (private heaps)       SHARED REDIS (one copy)

           +------+                                +------+
           |  LB  |                                |  LB  |
           +------+                                +------+
       +------+------+                     +------+------+
       |      |      |                     |      |      |
       v      v      v                     v      v      v
    +----+ +----+ +----+                +----+ +----+ +----+
    | R1 | | R2 | | R3 |                | R1 | | R2 | | R3 |
    |----| |----| |----|                +----+ +----+ +----+
    |LRU | |LRU | |LRU |                   |      |      |
    |k=v1| |k=v1| |k=v0|                   +------+------+
    +----+ +----+ +----+                          |
       |      |      |                            v
       | miss | miss | miss                 +-----------+
       v      v      v                    |   Redis   |
    +--------------------+                |  k = v1   |
    |         DB         |                +-----------+
    +--------------------+                      | 1 miss
                                                v
                                          +-----------+
                                          |    DB     |
                                          +-----------+

  3 copies of k; up to 3 DB misses     1 copy of k; 1 DB miss per key
  per TTL window. R3 still stale.      per TTL window. One DEL clears it.

Three replicas with private LRU heaps each hold their own copy of key k, miss separately, and can disagree (R3 still has v0), while replicas sharing one Redis see a single copy and a single miss.

Cache invalidation strategies and their guarantees

Three strategies cover most monoliths, and they differ mainly in what happens when something goes wrong.

TTL-only is the simplest to get right because nothing can be lost. You pay for that by accepting stale data for the full TTL.

Delete on write has a race. Suppose reader A misses the cache and loads the old row. Writer B commits and deletes the key. Then A writes its old value back into the cache. That entry stays stale until the next write or expiry. To narrow the window, delete only after the transaction commits, never before. Some teams also issue a second, delayed delete.

In-process caches need the delete to reach every replica. Redis pub/sub is fire-and-forget. A subscriber that is disconnected or reconnecting when a message is published never receives it, and Redis does not replay it. Each replica should still evict its own entry immediately on write. A short TTL on every local entry remains mandatory, because it caps how long a missed message can leave stale data in place. A durable message bus with consumer acknowledgements gives stronger delivery, but you then have more infrastructure to run.

Copyimport sqlite3
import threading

import redis
from cachetools import TTLCache

r = redis.Redis(host="redis.internal", port=6379)
CHANNEL = "cache-invalidate"

# Short TTL is the backstop: it caps how long a lost invalidation can leave stale data.
local = TTLCache(maxsize=10_000, ttl=30)
lock = threading.Lock()  # TTLCache is not thread-safe


def update_user_email(conn: sqlite3.Connection, user_id: int, email: str) -> None:
    key = f"user:{user_id}"
    with conn:  # sqlite3 connection context manager commits on successful exit
        conn.execute("UPDATE users SET email = ? WHERE id = ?", (email, user_id))
    # Invalidate only after commit, so readers cannot reload the pre-write row.
    r.delete(key)              # shared Redis cache
    with lock:
        local.pop(key, None)   # this replica, immediately
    r.publish(CHANNEL, key)    # other replicas; at-most-once, no replay


def listen_for_invalidations() -> None:
    pubsub = r.pubsub(ignore_subscribe_messages=True)
    pubsub.subscribe(CHANNEL)
    # Messages published while this subscriber is disconnected are lost for good.
    # The local TTL above is what eventually clears those entries.
    for msg in pubsub.listen():
        key = msg["data"].decode()
        with lock:
            local.pop(key, None)  # also receives our own publishes; harmless


threading.Thread(target=listen_for_invalidations, daemon=True).start()

A write path that commits first, then deletes the Redis key, evicts the local entry and publishes the key. A pub/sub listener on each replica evicts its own copy, and the short local TTL covers any messages the listener misses.

Sizing, memory, and failure modes

Bounding the local cache

An in-process LRU needs a hard limit, and you can set it by entry count or by estimated bytes. A count limit is cheap and predictable to enforce, but it says nothing about memory: 10,000 entries could take 2 MB or 2 GB depending on what you store. A byte limit tracks the real constraint, but you have to estimate each entry's size with a weigher function, and that estimate drifts as object shapes change. Use bytes when value sizes vary widely and count when they are uniform. Either way, an unbounded map is not a cache. It is a memory leak that grows with key cardinality until the process dies.

In managed runtimes, a large cache also costs you in garbage collection. Long-lived cached objects get promoted to the old generation, which inflates the heap and gives the collector more live data to scan and copy. That lengthens pause times on every replica. For example, you might cap the local cache at 50,000 entries with a five-minute TTL. On the Redis side, you might set maxmemory to a fixed budget with maxmemory-policy allkeys-lru. Without an explicit limit and eviction policy, Redis can grow until the host is under memory pressure or reject writes when it fills.

Operating Redis

With Redis, you take on operational work:

• Decide whether to persist with snapshots, an append-only file, or not at all. A pure cache often needs no persistence.
• Monitor memory use, the eviction rate, and the hit ratio.
• Plan for failover.

Wrap every call in a short timeout and fall back to the database when Redis is slow or unreachable.

Never let Redis quietly become the source of truth. If some data exists only there, a cache outage becomes data loss or a correctness bug.

Copy# redis.conf for a cache-only instance:
#   maxmemory 2gb
#   maxmemory-policy allkeys-lru   # evict least-recently-used keys instead of rejecting writes
#   save ""                        # no RDB snapshots: a pure cache can be rebuilt from the DB
#   appendonly no
import json
import redis
from cachetools import TTLCache

# Bounded by entry count: least-recently-used entries are evicted past 50k, and entries expire after 5 min.
# For a byte bound on bytes values, use TTLCache(maxsize=64 * 1024 * 1024, ttl=300, getsizeof=len).
local = TTLCache(maxsize=50_000, ttl=300)

# Short timeouts so a slow Redis degrades to a database read instead of stalling requests.
r = redis.Redis(host="redis.internal", port=6379,
                socket_timeout=0.05, socket_connect_timeout=0.05)

def get_product(product_id, db):
    key = f"product:{product_id}"
    value = local.get(key)
    if value is not None:
        return value
    try:
        cached = r.get(key)
    except redis.exceptions.RedisError:
        cached = None  # Redis is slow or down: treat it as a miss
    if cached is not None:
        value = json.loads(cached)
    else:
        value = db.load_product(product_id)  # the database remains the source of truth
        try:
            r.set(key, json.dumps(value), ex=600)
        except redis.exceptions.RedisError:
            pass  # a failed cache write must not fail the request
    local[key] = value
    return value

A read-through lookup with a count-bounded local TTL/LRU cache in front of Redis, which is capped by maxmemory with allkeys-lru eviction and wrapped in short timeouts that fall back to the database.

Cold starts, deploys, and stampede protection

An in-process cache lives and dies with its process. Every restart, crash, or rolling deploy starts a replica with an empty cache, and for the first minutes every request that would have been a hit goes to the database. With a rolling deploy across several replicas, this happens repeatedly: each new instance warms itself independently, so the database absorbs one cold-cache burst per replica. Redis runs outside the deploy cycle. A new release finds the same warm keys the old one left, which makes deploys uneventful from the database's point of view.

Both models share a second risk, the stampede. A hot key expires, hundreds of concurrent requests miss at the same moment, and all of them run the same expensive query.

In-process: per-key request coalescing

Keep a map of in-flight loads keyed by cache key. The first request that misses starts the load and registers a shared future or promise. Later requests for the same key wait on that future and do not query the database. When the load finishes, remove the entry, whether it succeeded or failed, so errors are not cached by accident. This caps database load at one query per key per replica.

Redis: locks or early refresh

Coalescing inside one process does not coordinate replicas. To coordinate them, take a short lock with SET lock:key token NX PX 5000. The winner recomputes the value and writes it, and the other replicas briefly serve the stale value or retry. The expiry ensures a crashed holder cannot block the key forever. Release the lock only if the stored token still matches, using a small Lua script, so a slow holder never deletes a lock that another replica now owns.

An alternative is probabilistic early refresh. Each reader refreshes slightly before expiry with a probability that rises as the deadline approaches. Recomputation is spread across time and readers, and you need no lock at all.

Alternatives: the two-tier cache and other options

The two-tier cache

A two-tier cache puts a small in-process LRU, with a TTL of a few seconds, in front of Redis. Reads check local memory first, then Redis, then the database. This works for hot, read-heavy keys such as feature flags, pricing tables, or a popular product page, where thousands of requests per second would otherwise make the same network round trip. The local tier absorbs that traffic.

The cost is a second source of staleness. When a write updates Redis, every replica can still serve its local copy until that copy's TTL expires. Redis-side invalidation does not reach the local tier unless you add a broadcast, such as Redis pub/sub, and a dropped message leaves a replica stale for the full TTL. Use two tiers only when you can state the staleness bound, and the business accepts it.

Other options

Try database tuning first. A cache layered over an unindexed query hides the problem without fixing it.

Decision checklist, cost, and engineering time

Start with in-process LRU if you run one or a few replicas and readers tolerate data that is a few seconds stale. It adds no network hop and no new infrastructure. Avoid it when users must see the same value on every replica right after a write, or when the working set competes with request handling for heap and drives up GC pressure.

Redis brings its own costs: provisioning, memory monitoring, eviction policy tuning, an on-call rotation that now includes another stateful system, and code paths for timeouts and outages. Per-replica caches have a less visible cost. Bugs where one replica serves stale data are intermittent, hard to reproduce, and expensive to debug. If your invalidation story is weak and you run many replicas, that debugging time can exceed the cost of operating Redis.

Measure on your own traffic before you commit. Ship an in-process cache behind a small interface, then record hit rate, database query rate, and p99 latency per replica. If the hit rate is high and database load stays flat as you add replicas, you are done. If hit rates fall as replicas multiply, because each replica warms its own copy, you have evidence for Redis. You also get a concrete baseline to evaluate it against, rather than adopting a distributed cache on speculation.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to in-process cache vs redis, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.