What are sticky sessions, and why do they become an operational problem?
Sticky sessions, also called session affinity, are a load balancer setting that sends every request from a given user to the same backend instance. The balancer usually does this with a cookie it sets on the first response, or by hashing the client IP. The setting exists because the application keeps session state in process memory: the logged-in user, the cart, CSRF tokens, multi-step form progress. If a request lands on a different instance, that instance has no record of the session, and the user is logged out.
Affinity works until you need to move users between instances, and production systems need to do that regularly.
- Uneven load. The balancer places a user when the session starts and never moves them. A few heavy users, or many users behind one corporate NAT under IP hashing, can saturate one instance while others sit idle.
- Lost sessions. When an instance crashes, every session in its memory is gone. Draining an instance for maintenance has the same effect unless you wait for its sessions to expire.
- Slow deploys. Rolling restarts either log users out or wait out long drain periods. That pressure pushes teams toward less frequent, larger releases.
- Autoscaling that cannot rebalance. New instances only receive new sessions. Existing users stay on the overloaded instances that triggered the scale-out.
Moving session state into a shared store such as Redis lets any instance serve any request, which makes the web tier stateless. This guide treats that move as a data-migration and failure-mode problem rather than a configuration toggle. It is written for teams running a monolith behind a load balancer with in-process sessions.
How does it work? Redis-backed sessions in a stateless web tier
With sticky sessions, the load balancer pins each client to one instance, usually through an affinity cookie or a hash of the client IP. Session state lives in that process's heap. A read is a pointer dereference, and the state disappears when the process does.
Before and after
Before: client → load balancer (affinity) → instance A → session in A's memory
After: client → load balancer (any instance) → instance A, B or C → session in Redis
In the new model the cookie holds only an opaque, random session ID. It should be long enough that it cannot be guessed, and it should carry no user data. On each authenticated request, the app reads that ID, fetches the session from Redis by key, and deserializes it. If the handler changes the session, the app serializes it and writes it back with a TTL. Any instance can serve any request because none of them owns the state.
What each request now costs
An in-memory lookup becomes a network round trip plus deserialization, and that cost lands on the critical path of every authenticated request. A changed session adds a write. Sliding expiration also needs a TTL refresh, which is a write even when nothing else changed. Write only dirty sessions, and refresh the TTL only when it falls below a threshold.
Redis load tracks request rate, not user count. Illustrative estimate: assume 2,000 authenticated requests per second, with 10% modifying the session. That produces about 2,000 reads and 200 writes per second. Double the traffic and both numbers double.
BEFORE: sticky sessions
+----------+ +----------------+ +------------------------+
| Client |--->| Load balancer |--->| Instance A |
| | | affinity: pins | | session in A's heap |
+----------+ | client to A | | lost if A restarts |
+----------------+ +------------------------+
AFTER: stateless web tier + Redis
+----------+ +----------------+ +------------+
| Client | | Load balancer |--->| Instance A |---+
| cookie: |--->| routes to any | +------------+ |
| sid=<id> | | instance |--->| Instance B |---+
+----------+ | | +------------+ |
| |--->| Instance C |---+
+----------------+ +------------+ |
|
every authenticated request: GET sess:<id> |
session changed: SET sess:<id> EX <ttl> v
+----------------------------+
| Redis |
| sess:<id> -> session blob |
| expires via TTL |
+----------------------------+
With sticky sessions, the load balancer pins each client to one instance that holds its session in memory. In the stateless tier, any instance reads the session from Redis by an opaque cookie ID and writes it back only when it changes.
When should you move to Redis sessions, and when should you avoid it?
Move when session loss has become a recurring operational problem. If you deploy several times a day, every rolling restart logs out or corrupts the carts of whoever was pinned to the drained instances. Autoscaling, spot or preemptible capacity, and node replacement all have the same effect: session affinity assumes instances live long enough to outlast their users' sessions. If you can trace incidents or support tickets to "instance died, users lost state," externalizing sessions removes that failure class.
Delay or skip it when the fleet is small and stable, deploys are infrequent, and nobody can operate Redis as a hard dependency. Once sessions live in Redis, a Redis outage becomes a login outage for everyone, not a partial loss. If your session holds little more than a user ID and a few flags, a signed or encrypted cookie may serve you better.
| Option | Revocation | Size limit | Latency | Operational burden | Security exposure |
|---|---|---|---|---|---|
| Sticky sessions | Easy, per instance | Instance memory | None added | Low, but fragile on churn | Data stays server-side |
| Signed/encrypted cookie | Hard without a server-side denylist | About 4 KB per cookie | None added, but sent on every request | Key rotation only | Payload sits on the client; replay until expiry |
| Redis | Immediate delete | Practically large | One network round trip | New critical service | Server-side; Redis access must be locked down |
| Database table | Immediate delete | Large | Slower, adds write load | Uses existing database | Server-side; adds churn to a shared store |
A database-backed table is often a reasonable middle step: if your read and write rates fit comfortably within your primary database's headroom, it avoids adding a new critical service.
Who should use Redis-backed sessions?
This migration fits teams that meet most of these conditions:
- You run a monolith on more than one instance behind a load balancer, and the framework's default session store keeps state in process memory or on local disk.
- Instances are replaced often. You deploy frequently with rolling restarts, autoscale on load, use spot or preemptible capacity, or rely on an orchestrator that reschedules workloads onto new nodes.
- Sessions carry state users would notice losing, such as authentication, carts, multi-step forms or in-progress edits, so a forced logout costs real work.
- You need server-side revocation. Security or compliance requirements call for logging a user out everywhere at once, which a Redis delete handles and a client-side cookie does not.
- Someone can own Redis as a critical dependency. That means monitoring, alerting, capacity planning and a failover runbook, because a Redis outage becomes a login outage.
It is a poor fit for a single-instance application, for systems that already authenticate each request with a self-contained token and keep no server-side session, and for teams whose session payload is small enough to live in a signed or encrypted cookie. Teams without the capacity to operate another stateful service should consider the database-table option first.
How do you prepare session data before externalizing it?
Start by dumping a sample of live sessions and listing every key with its size. Most monoliths accumulate cached query results, shopping-cart snapshots, rendered fragments and whole ORM objects. In process memory these cost almost nothing. Once sessions move to Redis, every byte crosses the network on every request. Move cached data to a proper cache with its own keys, keep durable state such as carts in the database, and set a per-session size budget, for example a few kilobytes. Enforce the budget with a logged warning so regressions show up.
Serialization
Use JSON with an explicit schema version, such as {"v":2,"user_id":4812,"roles":["editor"],"csrf":"…","created_at":1767225600}. Language-native object serialization ties the stored bytes to class definitions. If you rename a field or class, sessions written by the previous deploy fail to load mid-rollout, when old and new code are both running. Native deserializers also reconstruct arbitrary types, so anyone who can write to the store can potentially execute code. A version field lets new code read old payloads and upgrade them.
Keys and expiry
Generate IDs from a CSPRNG with at least 128 bits of entropy, and namespace them: sess:<id>. Write with SET sess:<id> <json> EX 1800, read with GET, and refresh with EXPIRE sess:<id> 1800.
- Sliding expiration renews the TTL on each request, so the TTL tracks the idle timeout. Each renewal is a write, which turns read-only traffic into writes. To reduce the cost, refresh only when the remaining TTL falls below a threshold.
- Absolute expiration stores
created_atin the payload and rejects any session older than the hard limit, however active it has been.
import json, logging, secrets, time
import redis
r = redis.Redis(host="localhost", port=6379, decode_responses=True)
IDLE_TTL = 1800 # seconds; matches the application idle timeout
ABSOLUTE_MAX = 8 * 3600 # hard session lifetime, however active the user is
REFRESH_BELOW = 1200 # renew only when less than this remains, to limit writes
SIZE_BUDGET = 4096 # bytes; over-budget writes are logged so regressions show up
def save(sid, data):
payload = json.dumps({**data, "v": 2}, separators=(",", ":"))
if len(payload.encode()) > SIZE_BUDGET:
logging.warning("session over budget: user=%s bytes=%d", data.get("user_id"), len(payload.encode()))
r.set(f"sess:{sid}", payload, ex=IDLE_TTL) # SET sess:<id> <json> EX 1800
def create(user_id, roles):
sid = secrets.token_urlsafe(32) # 256 bits from the OS CSPRNG, opaque to clients
save(sid, {"user_id": user_id, "roles": roles,
"csrf": secrets.token_urlsafe(32), "created_at": int(time.time())})
return sid
def load(sid):
key = f"sess:{sid}"
raw = r.get(key) # GET sess:<id>
if raw is None:
return None # expired or never existed
data = json.loads(raw)
if data.get("v") == 1: # older deploy stored a single role; upgrade in place
data["roles"] = [data.pop("role")]
save(sid, data) # rewrite as v2; this also resets the TTL
if time.time() - data["created_at"] > ABSOLUTE_MAX:
r.delete(key) # absolute expiration: reject regardless of activity
return None
if r.ttl(key) < REFRESH_BELOW:
r.expire(key, IDLE_TTL) # sliding expiration, but not a write on every request
return data
A redis-py session store that writes versioned JSON under a namespaced, high-entropy key with SET EX, upgrades old payloads on read, enforces an absolute lifetime, and refreshes the sliding TTL with EXPIRE only when the remaining TTL falls below a threshold.
How do you handle concurrent requests that race on the same session?
With sticky sessions, every request for a session landed on the same process, and many in-process session stores serialized access to a session or mutated a shared in-memory object. Once requests fan out across instances, that implicit coordination disappears. Two browser tabs, a page that fires parallel AJAX calls, or a client retrying a slow request can now hit different instances at the same moment. Each one loads the session, changes something, and writes it back.
Whole-blob overwrite vs. per-field updates
If the session is stored as a single serialized string, every write replaces the entire value. Suppose request A reads the blob and adds an item to cart, while request B reads the same blob and sets flash_message. Each request writes the full blob back with SET. Whichever finishes last wins, and the other request's change is lost. No error is raised, so this tends to surface as hard-to-reproduce bugs such as "my cart emptied itself."
Storing the session as a Redis hash narrows the race. Each request writes only the fields it changed, using HSET session:<id> cart <value>, so A and B touch different fields and both changes survive. Two writers on the same field still race, but the blast radius shrinks to that one field.
Two habits reduce exposure further:
- Write only on change. Track dirty fields and skip the write entirely for read-only requests. Most requests then cannot clobber anything.
- Keep the session minimal. Store identity and a few small flags in the session. Put carts, drafts and preferences in your primary database, where transactions or row-level locking already handle concurrent writes.
import json
import redis
r = redis.Redis(host="localhost", port=6379, decode_responses=True)
TTL = 1800 # session idle timeout in seconds
def add_to_cart_blob(sid, item):
data = json.loads(r.get(f"session:{sid}") or "{}")
data.setdefault("cart", []).append(item)
r.set(f"session:{sid}", json.dumps(data), ex=TTL) # rewrites every field
class Session:
def __init__(self, sid):
self.key = f"session:{sid}"
self.data = r.hgetall(self.key)
self.dirty = set()
def get(self, field):
return self.data.get(field)
def set(self, field, value):
self.data[field] = value
self.dirty.add(field)
def save(self):
if not self.dirty:
return # read-only request: skip the write, nothing to clobber
pipe = r.pipeline()
pipe.hset(self.key, mapping={f: self.data[f] for f in self.dirty})
pipe.expire(self.key, TTL) # refresh idle timeout
pipe.execute()
s = Session("3f9a2c7e")
s.set("flash_message", "Profile updated")
s.save()
Contrasts a whole-blob SET that loses concurrent updates with a Redis hash session that tracks dirty fields and writes only those with HSET, skipping the write entirely on read-only requests.
How do you cut over from sticky sessions to Redis without downtime?
Run the cutover as four separate deploys. Each one can be reversed without touching the others.
Phase 1: dual-read with write-through
On each request, the middleware reads the session ID from the cookie and looks it up in Redis. On a miss, it loads the local session and copies it into Redis. When the response is sent, it writes the session to both Redis and local storage. Keep affinity on, so the local copy stays current as a fallback.
- Verify: the Redis hit rate climbs toward 100% as active users return, and error rates and latency stay flat.
- Roll back: turn off the Redis read with a flag. Local data is still current.
Phase 2: stop local writes
- Verify: local fallback reads drop to zero for at least one full session lifetime.
- Roll back: re-enable local writes. Sessions created since this phase exist only in Redis, so roll back quickly or accept that those users will have to log in again.
Phase 3: remove affinity at the load balancer
- Verify: requests that land on a different node keep their session, and node drains no longer cause logouts.
- Roll back: re-enable affinity. This is harmless because Redis is already authoritative.
Phase 4: delete the dual-read code
Remove the fallback path and the feature flag so they don't become permanent dead weight.
Is a forced logout acceptable?
For internal tools, usually yes. In that case, skip phases 1 and 2: switch to Redis and remove affinity in one deploy, and every user logs in once. For checkout or long-form editing flows, a forced logout loses real work, so build the dual-read path.
Illustrative estimate (assumed numbers, not a benchmark): break-even months = (hours to build + hours to remove dual-read) ÷ hours saved per month on drains, deploys and scaling. Assume 24 hours to build, 6 to remove, and 10 hours saved per month: (24 + 6) ÷ 10 = 3 months.
import json
import redis # redis-py client
r = redis.Redis(host="sessions.internal", port=6379)
SESSION_TTL = 1800 # seconds; match the existing local session lifetime
def load_session(sid, local_store, flags):
if flags.read_redis: # Phase 1 on; turning it off is the Phase 1 rollback
raw = r.get(f"sess:{sid}")
if raw is not None:
metrics.incr("session.redis_hit")
return json.loads(raw)
data = local_store.get(sid) # fallback: the sticky node still has it
if data is not None:
metrics.incr("session.local_fallback") # must stay at zero before Phase 4
if flags.read_redis:
# Backfill so the next request hits Redis on any node
r.set(f"sess:{sid}", json.dumps(data), ex=SESSION_TTL)
return data
def save_session(sid, data, local_store, flags):
# Write-through: Redis always gets the latest copy, refreshing the TTL
r.set(f"sess:{sid}", json.dumps(data), ex=SESSION_TTL)
if flags.write_local: # Phase 2 turns this off; re-enable to roll back
local_store.set(sid, data)
def session_middleware(request, next_handler, local_store, flags):
sid = request.cookies.get("session_id")
request.session = (load_session(sid, local_store, flags) if sid else None) or {}
response = next_handler(request)
if sid:
save_session(sid, request.session, local_store, flags)
return response
Pseudocode for the dual-read and write-through session middleware. The read_redis and write_local flags map to the Phase 1 and Phase 2 deploys and their rollbacks, and the local_fallback counter is the signal you check before deleting the fallback path in Phase 4.
How can it save engineering time?
The savings come from removing work that exists only because instances own user state:
- Deploys stop waiting on drains. Rolling restarts can replace instances as soon as in-flight requests finish, instead of waiting for pinned sessions to expire or scheduling releases for low-traffic windows to limit logouts.
- A class of incidents goes away. "Users logged out after a deploy" and "cart emptied when a node died" no longer need investigation, customer-support follow-up or postmortems.
- Scaling needs less manual intervention. New instances take traffic from existing users right away, so nobody has to restart hot nodes or tune affinity to push load off them.
- Infrastructure changes get simpler. Node replacement, OS patching and moving to preemptible capacity become routine operations rather than coordinated events.
- Session bugs are easier to debug. An engineer can inspect a user's session directly with
GET sess:<id>orHGETALL session:<id>instead of working out which instance holds it and attaching to that process. - Load testing and local reproduction are more realistic, because requests no longer need to be pinned to one instance for the application to behave correctly.
Those savings are offset by new work: operating Redis, monitoring it, and handling its failure modes, covered in the next section.
Illustrative estimate (assumed inputs, not a measured result): assume a team does 20 rolling deploys per month, and each one costs about 15 minutes of an engineer's attention spent waiting on drains and watching for logout complaints. That is 5 hours. Assume 2 session-loss incidents per month at 2.5 hours each for investigation and follow-up, which adds another 5 hours. The total is about 10 hours per month, which is the figure assumed in the break-even calculation above. Substitute your own deploy frequency, drain time and incident history to get a number for your team.
Running Redis for sessions: sizing, failure modes and hardening
Sizing memory
Illustrative estimate (all inputs assumed): 200,000 active sessions × 4 KB average payload × 1.5 overhead factor for keys, TTL metadata and allocator fragmentation ≈ 1.2 GB. Provision about twice that, because replication buffers and fork-based persistence also consume memory.
Eviction policy
Every session key carries a TTL, so volatile-lru and allkeys-lru will evict live sessions under memory pressure. From the user's side, this looks like random logouts with no errors in the logs. Use noeviction so that writes fail loudly, and alert well before that point.
| Policy | Under memory pressure | Visible symptom |
|---|---|---|
allkeys-lru / volatile-lru |
Evicts active sessions | Silent random logouts |
noeviction |
Rejects writes | Errors you can alert on |
Persistence, replication and placement
Replication is asynchronous, so a failover can drop the most recent writes, such as a fresh login or a cart update. AOF with everysec limits loss after a restart to roughly one second of writes. Place Redis in the same zone as the app tier, because every request now pays a network round trip for its session.
Fail-closed or degraded mode
If Redis is down or slow, decide ahead of time what happens. Authenticated routes should fail closed. Anonymous pages can render without a session.
Hardening
- Use pooled connections with short connect and command timeouts.
- Require TLS and authentication.
- Store IDs in the session, not personal data or tokens.
- Monitor memory, evictions, latency and connection counts.
- Rehearse the failover runbook.
Post-migration checklist for hidden local state
- In-memory caches that assume one user stays on one node
- CSRF token stores held in process memory
- File uploads written to local disk between requests
- Websockets that still rely on load balancer session affinity
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to sticky sessions to redis sessions, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.