In-Process Cache vs Redis for Monoliths: Staleness and Invalidation

In-process cache vs Redis: what each one is and where the cost goes

An in-process cache lives on the application's heap. A hit is a hash lookup that returns a reference to an object already in memory, with no network hop and no serialization. Redis is a separate server that every instance of the monolith shares. A hit there costs a network round trip, serialization on write, deserialization on read, and a dependency that can be slow or unavailable.

Illustrative estimate (assumed numbers, not a benchmark). Assume a same-zone round trip of 0.5 ms, 20 µs to deserialize a 2 KB payload, and 10 cache reads per request.

added latency per request = reads × (RTT + server time + deserialize)

That works out to 10 × (0.5 + ~0.01 + 0.02) ms ≈ 5.3 ms for Redis. The in-process cache adds effectively nothing by comparison. Substitute your own RTT, payload size and read count.

Raw lookup speed is the wrong place to start the comparison. With N instances, an in-process cache holds N independent copies of each entry. If one instance handles a write, the other N−1 instances keep serving the old value until their TTL expires or an invalidation message reaches them. The question to answer first is how long each kind of data is allowed to be stale. That staleness budget drives every decision in the rest of this article.

Cache invalidation strategies and the consistency each one actually gives you

The delete-on-write race. Reader A misses the cache and loads row v1 from the database. Writer B commits v2 and deletes the key. A, still holding v1, writes it into the cache. v1 is now cached until the TTL expires. A delayed double-delete, which repeats the delete a few hundred milliseconds later, removes most of these entries. It does not remove all of them, because a slow reader can take longer than the delay. Versioning narrows the window further, but some gap remains.

Pub/sub is fire-and-forget. Redis does not store or replay pub/sub messages. An instance that is reconnecting, in a GC pause long enough to drop its connection, or restarting will never receive invalidations sent during that gap. The TTL is the only thing that bounds its staleness, so it cannot be removed.

Generation keys. Store a counter such as gen:user:42 = 7 in Redis and cache data under user:42:v7. A write increments the counter to 8. Readers then build user:42:v8, miss, and load fresh data. Nothing needs to be deleted, because the v7 entry is never read again and expires through its TTL. The cost is one extra Redis read per lookup.

Copyimport json
import redis

r = redis.Redis(host="localhost", port=6379, decode_responses=True)
DATA_TTL = 300  # seconds; still bounds staleness if anything goes wrong

def gen_key(user_id: int) -> str:
    return f"gen:user:{user_id}"

def current_generation(user_id: int) -> int:
    # No TTL on the counter: if it expired and reset to 0, later INCRs
    # could land on an old generation whose cached entry still exists.
    return int(r.get(gen_key(user_id)) or 0)

def get_user(user_id: int) -> dict:
    gen = current_generation(user_id)          # the extra Redis read per lookup
    data_key = f"user:{user_id}:v{gen}"
    cached = r.get(data_key)
    if cached is not None:
        return json.loads(cached)
    user = load_user_from_db(user_id)
    # If a write bumps the generation while we load, this lands under the
    # old generation's key, which no reader will build again.
    r.set(data_key, json.dumps(user), ex=DATA_TTL)
    return user

def update_user(user_id: int, fields: dict) -> None:
    save_user_to_db(user_id, fields)           # commit first
    # Then move readers to a fresh key. Nothing is deleted; the old
    # entry expires through DATA_TTL.
    r.incr(gen_key(user_id))

def load_user_from_db(user_id: int) -> dict:
    return {"id": user_id, "name": "placeholder"}  # stand-in for a real query

def save_user_to_db(user_id: int, fields: dict) -> None:
    pass  # stand-in for a real UPDATE + COMMIT

A generation-key cache in Python with redis-py: each write increments gen:user:. Readers then build a new versioned key and miss, so no delete is needed, and the TTL clears orphaned entries.

How to implement an in-process cache with Caffeine

Each Caffeine setting answers a specific design question. The configuration example that accompanies this section uses maximumSize, expireAfterWrite, refreshAfterWrite, recordStats(), and a loader passed to build.

Bounding and expiring entries

maximumSize caps the entry count. When the cache is full, Caffeine evicts entries that are unlikely to be reused. Without a cap, the cache grows until the heap runs out.

expireAfterWrite limits how old a value can be. expireAfterAccess resets that clock on every read, so a frequently read key may never expire. It can keep serving a value from hours ago while its TTL looks short. Use write-based expiry whenever staleness matters.

refreshAfterWrite should be shorter than the expiry. After it elapses, the next read returns the current value and triggers an asynchronous reload. Callers do not block on the reload, but they can get one stale read per refresh.

Loading and statistics

Building with a loader produces a LoadingCache. Concurrent misses on the same key share one load, which protects the database from a burst of identical queries. This works only within one JVM. Ten instances can still issue ten loads for that key.

recordStats() exposes hit rate, load time, and eviction counts. Without these numbers you cannot tune size or TTL.

Memory and GC costs

Cache memory is multiplied by the number of instances. Cached objects live on the heap, so a large cache means a bigger heap and potentially longer GC pauses. In a monolith, the cache competes for memory with request handling, batch jobs, and everything else, which raises the risk of out-of-memory errors. Every deploy also starts each instance with an empty cache.

Copyimport com.github.benmanes.caffeine.cache.Caffeine;
import com.github.benmanes.caffeine.cache.LoadingCache;
import com.github.benmanes.caffeine.cache.stats.CacheStats;
import java.time.Duration;

public class ProductCache {

    // Placeholder types and repository; substitute your own.
    public record Product(long id, String name) {}
    public interface ProductRepository { Product findById(long id); }

    private final LoadingCache<Long, Product> cache;

    public ProductCache(ProductRepository repo) {
        this.cache = Caffeine.newBuilder()
            // Bound the entry count so the cache cannot grow until the heap runs out.
            .maximumSize(10_000)
            // Hard staleness limit: no value is served more than 10 minutes after it was loaded.
            // Write-based, so frequent reads cannot keep an old value alive.
            .expireAfterWrite(Duration.ofMinutes(10))
            // Shorter than expiry: after 1 minute the next read returns the current value
            // and triggers an async reload, so callers do not block on the database.
            .refreshAfterWrite(Duration.ofMinutes(1))
            // Collect hit rate, load time and eviction counts for tuning size and TTLs.
            .recordStats()
            // Passing a loader makes this a LoadingCache: concurrent misses on the same key
            // in this JVM share one load. Other instances still load independently.
            .build(repo::findById);
    }

    public Product get(long id) {
        return cache.get(id);
    }

    // Export these to your metrics system; cache.stats() returns a snapshot.
    public String statsSummary() {
        CacheStats s = cache.stats();
        return String.format("hitRate=%.3f evictions=%d avgLoadNanos=%.0f",
            s.hitRate(), s.evictionCount(), s.averageLoadPenalty());
    }
}

A Caffeine LoadingCache configured with a size bound, write-based expiry, a shorter refresh interval for asynchronous reloads, and recordStats() for reading hit rate, evictions and load time.

How a two-level cache works: in-process L1 in front of Redis L2

A two-level cache puts a small in-process cache (L1, such as Caffeine) in front of a shared Redis cache (L2): request → L1 (per instance) → L2 (Redis) → database. L1 handles hot keys without a network hop. L2 absorbs misses and keeps cold starts from reaching the database.

Read path

1. Check L1. On a hit, return.
2. Check L2. On a hit, write the value to L1 and return.
3. Load from the database, write to L2 and then L1, and return.

Write path

1. Commit the change to the database.
2. Delete the key from L2.
3. Publish an invalidation message, for example over Redis pub/sub, so every instance evicts the key from its L1.

The order matters because each step makes the next refill correct. If an instance evicts L1 before L2 is cleared, it can refill L1 from the stale L2 entry. If L2 is cleared before the database commits, a concurrent reader can repopulate L2 with the old row. Pub/sub delivery is fire-and-forget, so a disconnected instance never receives the message. A short L1 TTL, measured in seconds, sets an upper bound on how long a missed invalidation can serve stale data.

Costs

Stale data can now live in two places, and you have to reason about both. Debugging is harder because two instances can return different values for the same key at the same moment, and neither answer is wrong from that instance's point of view. L2 also couples serialization formats across instances. During a rolling deploy, old and new code read each other's entries, so include a schema version in the key (user:v3:42) to keep incompatible payloads apart.

Copyimport json, threading
import redis
from cachetools import TTLCache

# request -> L1 (per process) -> L2 (Redis) -> database
r = redis.Redis(host="cache.internal", port=6379)
l1 = TTLCache(maxsize=10_000, ttl=5)  # short TTL bounds staleness from lost invalidations
l1_lock = threading.Lock()            # TTLCache is not thread-safe; pub/sub runs on its own thread
CHANNEL = "cache-invalidate"

def key(user_id):
    return f"user:v3:{user_id}"  # schema version keeps old/new payloads apart during rolling deploys

def get_user(user_id):  # `db` is a placeholder for your data access layer
    k = key(user_id)
    with l1_lock:
        if k in l1:                     # 1. L1 hit: no network hop
            return l1[k]
    raw = r.get(k)                      # 2. L2 (shared Redis)
    if raw is not None:
        value = json.loads(raw)
    else:
        value = db.load_user(user_id)   # 3. database
        r.set(k, json.dumps(value), ex=300)  # populate L2 first...
    with l1_lock:
        l1[k] = value                   # ...then L1 on the way back
    return value

def update_user(user_id, fields):
    db.update_user(user_id, fields)     # 1. commit first so any refill reads the new row
    k = key(user_id)
    r.delete(k)                         # 2. clear L2 before L1s evict, so they cannot refill stale
    r.publish(CHANNEL, k)               # 3. fire-and-forget; the L1 TTL covers missed messages

def on_invalidate(message):
    with l1_lock:
        l1.pop(message["data"].decode(), None)

pubsub = r.pubsub()
pubsub.subscribe(**{CHANNEL: on_invalidate})
pubsub.run_in_thread(sleep_time=1.0, daemon=True)

A minimal two-level cache in Python: reads fall through L1, then Redis, then the database and fill both levels on the way back, while writes commit, delete from Redis, then broadcast an L1 eviction over pub/sub, with a 5-second L1 TTL as the backstop.

Stampedes, Redis outages, and cold starts

Caffeine's loading cache coalesces concurrent misses for the same key, so each instance issues one load. That protection stops at the process boundary. When a hot key expires in Redis, every instance misses at once, and forty pods mean forty identical queries hitting the database together. The stampede you need to worry about is cluster-wide, not per-instance.

Three mitigations handle different parts of the problem:

• Jittered TTLs. Add a random offset to each TTL, for example ±10%, so keys written together do not expire together.
• Early or probabilistic refresh. A reader refreshes the entry shortly before expiry, and the chance of refreshing rises as the deadline approaches. One caller rebuilds the entry while everyone else keeps reading the old value.
• Short-lived locks. SET lock:key token NX PX 3000 lets one instance rebuild the entry. The others wait briefly or serve stale data. Set the PX to more than a typical rebuild takes, so the lock cannot expire while a slow rebuild is still running.

Redis is a shared, critical dependency, so define its failure mode explicitly. Use client timeouts in the tens of milliseconds. Keep L1 serving entries past their TTL, marked stale, while Redis is unreachable. Put a semaphore in front of database loads so a failover or partition cannot push unbounded concurrency onto the database. Callers that cannot get a permit receive stale data or a fast error, not a queued query.

Rollouts cause cold starts: each new pod begins with an empty L1. Warm-up code that preloads known hot keys smooths this out, but it slows readiness, can load keys nobody requests, and becomes a stampede of its own if every pod warms at the same moment. Stagger the rollout, and warm from Redis, not from the database.

Copyimport random, threading, uuid
import redis

r = redis.Redis(host="redis.internal", port=6379, socket_timeout=0.05)  # fail fast, tens of ms
db_permits = threading.BoundedSemaphore(8)  # cap concurrent DB loads from this instance
l1 = {}  # key -> last known value, kept past TTL as a stale fallback

# Delete the lock only if we still own it, so we never release another instance's lock.
RELEASE = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) end return 0"

def jittered_ttl(base=300):
    return int(base * random.uniform(0.9, 1.1))  # +/-10% so co-written keys expire apart

def load_bounded(key, load_from_db):
    if not db_permits.acquire(timeout=0.05):
        raise RuntimeError("db load shed")  # fast error instead of a queued query
    try:
        return load_from_db(key)
    finally:
        db_permits.release()

def get(key, load_from_db):
    try:
        val = r.get(key)
        if val is not None:
            l1[key] = val
            return val
        token = uuid.uuid4().hex
        if not r.set(f"lock:{key}", token, nx=True, px=3000):  # PX > typical rebuild time
            return l1.get(key)  # another pod is rebuilding; serve stale (caller may retry briefly)
        try:
            val = load_bounded(key, load_from_db)
            r.set(key, val, ex=jittered_ttl())
            l1[key] = val
            return val
        finally:
            r.eval(RELEASE, 1, f"lock:{key}", token)
    except redis.exceptions.RedisError:
        # Redis slow, failing over, or partitioned: prefer stale L1, else a bounded DB load.
        return l1[key] if key in l1 else load_bounded(key, load_from_db)

A redis-py read path that combines jittered TTLs, a SET NX PX rebuild lock with owner-checked release, a semaphore bounding database loads, and a stale-L1 fallback when Redis is unavailable.

When to use in-process, Redis, or a two-level cache, and when to avoid caching

Choose the layer based on the kind of data, not on how fast you want reads to be.

In-process caching suits config and flags because they change rarely. If pods briefly hold different copies within a short TTL, nothing breaks. Redis suits sessions and counters because correctness depends on a single shared value. A rate limiter that counts per pod lets a client exceed its limit by a factor of the pod count. A two-level cache is worth its complexity only when Redis round trips show up in your latency profiles.

Match the option to your team:

• Small teams with little operational support: start with in-process caching, because it adds no infrastructure.
• Teams already running Redis with alerting and on-call: use Redis for shared state.
• Two-level caching: adopt it only if your team can operate the invalidation channel and measure staleness. Bugs there appear as intermittent wrong data that is hard to reproduce.

Before adding any cache, check the alternatives:

• Fix missing indexes and N+1 queries.
• Add read replicas for read volume, keeping in mind that replica lag is its own form of staleness.
• Use materialized views for expensive aggregates.
• Set HTTP caching headers or put a CDN in front of public responses.

Any of these can remove the need for an application cache entirely.

Observability, development cost, and engineering time

What to measure

Without these numbers you cannot tell whether a cache layer is helping or just adding moving parts:

• Hit ratio per level. Track L1 and L2 separately. A combined ratio can hide an L1 that rarely hits and only adds memory pressure.
• Eviction counts. Rising evictions alongside a falling hit ratio usually mean the cache is undersized or the key space is too broad.
• Invalidation message lag. Measure the time from the write to the moment the L1 entry is dropped on every node. This lag is your real staleness window, not the TTL you configured.
• Redis latency. Track p99, not the mean. A slow L2 can be worse than going straight to the database.
• Staleness incidents. Count bug reports traced to stale data. This is the metric product teams notice first.

Development and maintenance cost

Decide by staleness budget first

Start by stating how stale each kind of data is allowed to be. If 60 seconds is acceptable, a TTL-only in-process cache needs no invalidation code. If the answer is "never stale," caching may be the wrong tool. Making this decision first keeps you from building a two-level cache you don't need, and from fixing the same stale-data incident every quarter.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to in-process cache vs redis, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.