What is cache-aside with delete-on-write, and why is it the safe default?
Cache-aside puts your application code in charge of the cache. Redis never talks to the database. The application handles both paths.
Read path:
GETthe key from Redis.- On a hit, return the cached value.
- On a miss, query the database,
SETthe result with a TTL, and return it.
Write path:
- Update the database.
- Delete the key.
The next read misses and repopulates the cache from the current row.
Why delete instead of update
The tempting alternative is to write the new value into the cache after the database update. That breaks under concurrency. Suppose writer A commits version 1 and writer B commits version 2, but B's cache SET lands before A's. The cache now holds version 1 even though the database holds version 2, and it stays wrong until the TTL expires.
Deletes do not have this problem. They are idempotent and carry no payload, so two deletes in any order leave the same empty slot. The next reader then fetches whatever the database holds at that moment.
Deleting also keeps serialisation in one place. Only the read path decides what a cached object looks like. Write paths do not need to know the cached format, so they cannot drift out of step with it.
What it does not give you
Delete-on-write is not free of races, and the remaining ones come from the order of operations. That is the pattern behind most stale-data bugs: they usually trace back to the sequence of reads, writes and deletes, not to the choice of cache. Cache-aside gives you bounded staleness: the TTL caps how long a wrong value can survive. It does not give you strong consistency. Getting that would require locking or coordination between the cache and the database on every read and write. That coordination would cancel most of the latency you added Redis to save.
import json
import redis
# Placeholder connection; use your own host and credentials.
r = redis.Redis(host="localhost", port=6379, db=0)
TTL_SECONDS = 300 # upper bound on how long a stale value can survive
def cache_key(user_id: int) -> str:
return f"user:{user_id}"
def get_user(db, user_id: int) -> dict | None:
# Read path: try the cache first.
cached = r.get(cache_key(user_id))
if cached is not None:
return json.loads(cached)
# Miss: load from the source of truth.
row = db.fetch_user(user_id) # placeholder DB call
if row is None:
return None
# Serialisation lives only on the read path.
r.set(cache_key(user_id), json.dumps(row), ex=TTL_SECONDS)
return row
def update_user_email(db, user_id: int, email: str) -> None:
# Write path step 1: commit to the database first.
db.update_user_email(user_id, email) # placeholder DB call
# Step 2: delete, don't SET. Deletes are idempotent and carry no
# payload, so concurrent writers can't leave an older version cached.
r.delete(cache_key(user_id))
Cache-aside with redis-py: reads populate the cache with a TTL on a miss, and writes update the database and then delete the key instead of writing a new value.
Who should add a Redis cache, and when should you avoid it?
Start with measurement, not intuition. Profile production traffic and find the read paths that dominate database time: the product page query that runs on every request, the configuration lookup inside a hot loop. Caching pays off when a small set of entities absorbs a large share of reads and changes rarely, because each write invalidates the entry and forces a miss. An entity read a thousand times between writes keeps nearly every request off the database. One read once or twice between writes gains almost nothing.
Before writing cache code, record a staleness budget for each candidate entity: how long a reader can see an old value before it becomes a bug. "A product description can lag by 60 seconds" is a decision the team can review. A cache built without that number has no definition of correct behaviour.
Avoid caching write-heavy data, and avoid it for reads that must be strongly consistent, such as account balances or permission checks. A stale permission grants access that was already revoked. Also skip entities where the database query is already fast, because the gain won't justify the extra moving part.
Development cost is the less visible reason to hold back. Each cached entity adds a read path, a fallback path, and invalidation logic for every code path that mutates it. A missed invalidation in a rarely used admin endpoint is a typical source of stale-data bugs that only surface weeks later. Cache the few entities that matter and leave the rest uncached.
| Read/write ratio | Staleness tolerance | Cost of a stale read | Recommendation |
|---|---|---|---|
| High | Minutes | Low (cosmetic) | Cache |
| High | Seconds | Moderate | Cache with a short TTL |
| Low | Any | Any | Do not cache |
| Any | None | High (money, access control) | Do not cache |
How do race conditions cause stale reads in cache-aside?
The core race needs only two requests and one slow reader:
- Reader A misses the cache and reads row version v1 from the database.
- Reader A stalls, for example during a GC pause or on a slow network hop.
- Writer B updates the row to v2, commits, and deletes the key.
- Reader A resumes and SETs v1 into the cache.
The cache now holds v1 and no further writes are coming to remove it. It stays wrong until the next write or until the TTL expires.
Deleting the key before the commit makes this worse. Between the delete and the commit, every reader that misses reads the last committed row, which is still v1, and writes it back to the cache. The stale window therefore covers the entire transaction, including lock waits and slow statements, instead of a few milliseconds of bad timing. If the transaction rolls back, the delete achieved nothing. Readers have already repopulated the key, so you paid for a burst of misses and still got no guarantee in either outcome.
| Approach | Stale window | Effect of rollback | Residual risk |
|---|---|---|---|
| Delete before commit | Whole transaction | Cache repopulated anyway | High |
| Delete after commit | Slow reader's SET lands after the delete | Not applicable | Low but nonzero |
| After commit + TTL | Capped at TTL | Not applicable | Bounded |
| After commit + versioned key | Stale SET lands on an old key | Not applicable | Version lookup can itself be stale |
Deleting after the commit narrows the race but does not close it. A TTL limits how long a stale value can survive. Versioned keys, which embed a row version column in the key, send a late SET to a key that current readers no longer request. The race still exists if a reader looked up the version before the commit and then read the row data after it.
How do you tie cache invalidation to transaction commits?
Whichever key scheme you use, the timing of the DEL matters as much as the key. If you delete inside an open transaction, a concurrent reader can miss the cache, read the old committed row (your change is not visible to it yet), and write that old value back into Redis. Your transaction then commits, and the cache holds stale data until the TTL expires. Deleting before commit therefore invalidates too early, and a rollback after an early delete only wastes the reload it triggered.
The fix is to treat cache deletes as a side effect of a successful commit, not a step inside the transaction. A unit-of-work pattern handles this cleanly:
- When the transaction begins, the unit of work starts an empty set of pending keys.
- Each repository method that writes a row adds the affected keys to that set. This covers the entity key and any list or aggregate keys that include it. Nothing touches Redis yet.
- On commit, the unit of work commits the database transaction first. Only if that succeeds does it issue the deletes, ideally as one multi-key DEL or a pipeline.
- On rollback, it discards the set. No row changed, so no key should be invalidated.
Most frameworks expose an after-commit hook or callback for this. Use that hook rather than placing the delete after the commit call by hand, which is easy to get wrong in nested transactions.
One gap remains. If the process crashes, or the Redis call fails, between the commit and the DEL, the invalidation is lost. Nothing retries it, so the TTL is your only recovery. That is why every cached key needs a TTL, even when you invalidate explicitly.
import logging
log = logging.getLogger(__name__)
class UnitOfWork:
def __init__(self, conn, cache):
self.conn = conn # DB-API connection, transaction open until commit
self.cache = cache # Redis client; delete() takes multiple keys
self.pending = set()
def __enter__(self):
self.pending = set() # new transaction, no keys pending
return self
def invalidate_after_commit(self, *keys):
self.pending.update(keys) # record only; Redis is not touched yet
def __exit__(self, exc_type, exc, tb):
keys, self.pending = self.pending, set()
if exc_type is not None:
self.conn.rollback() # no row changed, so drop the keys
return False
self.conn.commit() # if commit raises, no DEL is sent
if keys:
try:
self.cache.delete(*keys) # one multi-key DEL after commit
except Exception:
# Lost invalidation: nothing retries it, only the TTL recovers
log.exception("cache invalidation failed: %s", sorted(keys))
return False
def rename_user(uow, user_id, name):
cur = uow.conn.cursor()
cur.execute("UPDATE users SET name = %s WHERE id = %s", (name, user_id))
# Entity key plus any list key that includes this user
uow.invalidate_after_commit(f"user:{user_id}", "users:active:list")
# with UnitOfWork(conn, redis_client) as uow:
# rename_user(uow, 42, "Ada")
A unit of work collects cache keys during the transaction, issues one multi-key DEL only after a successful commit, and discards the keys on rollback. A crash or Redis failure after the commit still leaves the key to expire by TTL.
Designing keys, TTLs and stampede protection
Keys built for invalidation
Namespace and version every key, for example app:v3:user:42. Bumping the version prefix on a schema change retires every old entry at once. Write paths should delete exact keys they can compute from the row they changed. KEYS and broad SCAN patterns do not belong in hot paths. KEYS blocks Redis while it walks the keyspace. SCAN is incremental, but it still visits every key to find a few.
Lists and aggregates are harder, because one row change affects many derived keys. Embed a generation counter in those keys, such as app:v3:team:7:gen:{n}:members. A write runs a single INCR on the team's counter, and readers build new keys from the new value. The trade-off is that old generations become orphans. Only their TTL and maxmemory eviction reclaim that space.
TTLs as a backstop
Derive the TTL from your staleness budget. It is the longest a lost invalidation can serve wrong data. Add random jitter, for example ±10%, so keys written together do not all expire at the same moment.
Illustrative estimate. Assume 50,000 hot keys, each refilled by a 20 ms query. A 60-second TTL causes about 830 refills per second and caps staleness at 60 seconds. A 600-second TTL causes about 83 refills per second but allows up to 10 minutes of staleness.
Stampede protection
When a hot key expires, coalesce concurrent misses so only one request queries the database. Across processes, take a short lock with SET lock:key token NX PX 2000. The other requests either wait, which adds latency to every waiting request, or receive the previous value while the lock holder refreshes it. Stale-while-refresh accepts that staleness by design, so keep the window inside your staleness budget.
import json, random, time, uuid
import redis
r = redis.Redis(host="localhost", port=6379, decode_responses=True)
FRESH_TTL = 60 # seconds; soft TTL taken from the staleness budget
GRACE = 30 # extra seconds a stale copy may be served during a refresh
# Delete the lock only if it still holds our token, so we never release another worker's lock.
UNLOCK = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) end return 0"
def members_key(team_id):
# The generation counter is embedded in every derived key for this team.
gen = r.get(f"app:v3:team:{team_id}:gen") or 0
return f"app:v3:team:{team_id}:gen:{gen}:members"
def invalidate_team(team_id):
# One INCR retires all derived keys. Old generations stay as orphans until TTL or eviction removes them.
r.incr(f"app:v3:team:{team_id}:gen")
def get_members(team_id, load_from_db, attempts=40):
key = members_key(team_id)
raw = r.get(key)
entry = json.loads(raw) if raw else None
if entry and entry["fresh_until"] > time.time():
return entry["value"]
token = str(uuid.uuid4())
if r.set(f"lock:{key}", token, nx=True, px=2000): # one worker refreshes
try:
value = load_from_db(team_id)
ttl = FRESH_TTL * random.uniform(0.9, 1.1) # +/-10% jitter spreads out expiries
doc = json.dumps({"value": value, "fresh_until": time.time() + ttl})
r.set(key, doc, ex=int(ttl + GRACE)) # the hard TTL is the backstop
return value
finally:
r.eval(UNLOCK, 1, f"lock:{key}", token)
if entry:
return entry["value"] # stale-while-refresh: staleness is accepted, bounded by GRACE
if attempts == 0:
return load_from_db(team_id) # stop waiting rather than hang
time.sleep(0.05) # cold miss: waiting adds latency to this request
return get_members(team_id, load_from_db, attempts - 1)
A redis-py sketch that combines a generation-counter key for a team's member list, a jittered soft TTL with a hard-TTL backstop, and a SET NX PX lock so that one worker refreshes while the others serve the stale copy or wait briefly.
What are the alternatives to delete-on-write?
Plain cache-aside with an after-commit DEL has one gap: if the process crashes, or Redis is unreachable, between the commit and the delete, the old value survives until its TTL expires. Each alternative closes a different gap, and each adds cost.
| Approach | Fixes | Does not fix | Cost |
|---|---|---|---|
| After-commit delete (baseline) | Stale entries after normal writes | Lost deletes on crash; read-populate races | Lowest |
| Write-through | Cold reads after writes | Concurrent writers can leave an older value cached; lost writes on crash | Every write path must build the cached shape |
| Delayed double-delete | A reader repopulating from a lagging replica | Lost deletes on crash | A scheduled second DEL; picking a delay longer than replica lag |
| Change-log (CDC) invalidation | Lost deletes, and writes that bypass the app | Replication lag delays invalidation | A log-tailing service, schema-change handling, on-call ownership |
| Outbox table | Lost deletes | Invalidation runs a little later than the commit | Extra table, relay worker, row cleanup |
Write-through fits read-heavy keys that writers already hold in full. Without versioning, two concurrent writers can put an older value in the cache last.
Delayed double-delete is a cheap patch for reads that go to replicas. Its correctness depends on the delay exceeding replica lag, and you can only estimate that lag.
CDC suits monoliths where scripts, migrations or other services write to the same tables. It reads the database's own commit log, so no write is missed. You do have to run and monitor that log-tailing service.
An outbox gets most of CDC's durability using only your existing database. The invalidation row commits atomically with the data, and a worker retries the DEL until it succeeds.
How do you roll out and monitor a Redis cache in a monolith safely?
Before any request is served from Redis, route every cache read, write and invalidation through one data-access module. If controllers or background jobs call Redis directly, each call site becomes its own invalidation rule, and you cannot audit them all. With a single module, the key format, TTL and delete-on-commit hook live in one place, and so does the flag check.
Roll out one entity at a time
Put each entity type behind its own flag. Start with something read-heavy and rarely written, such as product categories, before you cache orders or balances. For each entity, run in shadow mode first. The module reads from both cache and database, serves the database result, and logs any mismatch. When shadow mismatches stay at zero across normal traffic and deploys, switch that entity to serve from cache.
What to watch
Track hit and miss rates per entity. Also keep a sampled divergence check running after cutover: pick a small fraction of cache hits, re-read the row, and count mismatches. A rising mismatch count points to a missed invalidation path before users report it. Make each entity's flag a kill switch that on-call engineers can flip without a deploy.
Plan for a cold cache
After a Redis outage or failover, every read goes to the database at once. Load-test that scenario. If the database cannot absorb the load, add load shedding, a small in-process cache, or rate limiting on the hottest endpoints.
Finding a divergence in shadow mode costs one log query. The same bug found in production means reconstructing a race from incomplete traces.
import json, logging, random
import redis
r = redis.Redis(host="cache.internal", port=6379, socket_timeout=0.05)
log = logging.getLogger("cache")
# Per-entity mode read from your flag store: "off" | "shadow" | "on".
# Setting an entity to "off" is the kill switch and needs no deploy.
FLAGS = {"category": "on", "order": "shadow"}
TTL = {"category": 3600, "order": 60}
VERIFY_RATE = 0.01 # fraction of cache hits re-checked against the DB
def count(entity, event):
pass # emit a metric such as cache_events{entity, event}
def get(entity, key, load_from_db):
mode = FLAGS.get(entity, "off")
if mode == "off":
return load_from_db(key)
ckey = f"{entity}:v1:{key}"
try:
raw = r.get(ckey)
except redis.RedisError:
count(entity, "error") # Redis down: the DB takes this load
return load_from_db(key)
if raw is None:
count(entity, "miss")
row = load_from_db(key)
try:
r.set(ckey, json.dumps(row), ex=TTL[entity])
except redis.RedisError:
count(entity, "error")
return row
count(entity, "hit")
cached = json.loads(raw)
# Shadow mode checks every hit; after cutover, only a sample is checked.
if mode == "shadow" or random.random() < VERIFY_RATE:
row = load_from_db(key)
if row != cached:
count(entity, "divergence")
log.warning("cache divergence entity=%s key=%s", entity, key)
return row # the DB value is correct; serve it
return cached
def invalidate(entity, key):
# Called from the after-commit hook. It runs in every mode so the
# cache is already clean when an entity is switched to "on".
r.delete(f"{entity}:v1:{key}")
A single data-access module that sets the cache mode per entity with a flag (off, shadow, on), serves from the database during shadow reads, checks a sample of cache hits for divergence after cutover, and falls back to the database when Redis errors.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to cache invalidation strategies, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.