Scaling a Monolith Horizontally: Sessions, Uploads and Caches

What replica-safe means and why scaling a monolith horizontally breaks on local state

Scaling a monolith horizontally means running several identical copies of the same application behind a load balancer, rather than splitting it into services or buying a bigger machine. Teams do it for three reasons. If one replica dies, the others keep serving (availability). Request throughput grows by adding processes (capacity). Deploys can drain and replace replicas one at a time (zero-downtime deploys).

A replica-safe application gives the same result no matter which replica handles a request, and no matter how many replicas are running. Most monoliths fail this test because of assumptions baked in while they ran alone, not because of CPU or the load balancer. Common examples:

• A session lives in process memory.
• An upload lands on local disk.
• An in-memory cache is treated as the source of truth.
• A cron entry fires on every host that has it.

With one instance these assumptions hold. With two, a user logs in on replica A and appears logged out on replica B. Invalidating a cache on one replica leaves stale data on the other. A nightly invoice job runs twice.

Copy   BEFORE                         AFTER
+-----------------+          +---------------+
|   app process   |          | load balancer |
|  memory: cache, |          +-------+-------+
|    sessions     |          +-------+-------+
|  local disk:    |          |       |       |
|    uploads      |        [app1] [app2] [appN]
|  cron           |          |       |       |
+-----------------+          +-------+-------+
                             |       |       |
                         session  cache  object   queue
                          store          storage

This checklist is for backend and platform engineers whose monolith runs as a single instance and is about to get a second replica. It covers what to find, what to move out of the process and in what order.

Copy        BEFORE                          AFTER
       clients                         clients
           |                              |
           v                              v
+---------------------+     +---------------------------+
|     app process     |     |       load balancer       |
|                     |     +---------------------------+
| memory:             |         |         |         |
|   sessions, cache   |         v         v         v
| local disk:         |     +-------+ +-------+ +-------+
|   uploads           |     | app1  | | app2  | | appN  |
| cron:               |     +-------+ +-------+ +-------+
|   nightly jobs      |         |         |         |
+---------------------+         +---------+---------+
                                          |
  all state is local to         +---------+---------+----------+
  one process and host          |         |         |          |
                                v         v         v          v
                            +-------+ +-------+ +--------+ +-------+
                            |session| | cache | | object | | queue |
                            | store | |       | |storage | |       |
                            +-------+ +-------+ +--------+ +-------+

Before: one app process keeps sessions and cache in memory, uploads on local disk, and runs its own cron. After: N identical replicas behind a load balancer, all using a shared session store, cache, object storage and queue.

Step 1: Audit local state and reproduce bugs with two local replicas

Treat every piece of state the process keeps for itself as a bug until you can show it is safe. That means anything written to local disk, anything held in process memory across requests, any job scheduled in-process, and anything keyed to the hostname. Most of these are obvious once you search for them. The cases below are the ones teams usually miss, because each works perfectly on a single instance.

To reproduce these bugs, run two copies of the app on different ports. Point both at the same database. Put a reverse proxy in front configured for strict round-robin with no session affinity. Strict alternation matters: it guarantees that consecutive requests land on different replicas, so state bugs show up on every second request instead of rarely.

Then walk through three flows by hand:

• Login: a bounce back to the login page points to sessions held in local memory.
• Upload: a 404 when you fetch the file you just uploaded points to local disk storage.
• Checkout: a failed CSRF check or a duplicate order ID points to in-memory tokens or local ID counters.

Record each failure as it appears. The next sections fix them.

Copy# Run two copies of the app on ports 8001 and 8002, both using the same database.
# Start the proxy with: nginx -p "$PWD" -c nginx.conf
events {}

http {
  upstream app_replicas {
    # Round-robin is the default. There is no ip_hash, hash or sticky directive,
    # so a single tester's consecutive requests alternate between replicas.
    server 127.0.0.1:8001;
    server 127.0.0.1:8002;
  }

  server {
    listen 8080;

    # Large enough to exercise the upload flow.
    client_max_body_size 20m;

    location / {
      proxy_pass
      proxy_set_header Host $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;

      # Do not retry a failed request on the other replica.
      # A retry could mask the state bugs you are trying to find.
      proxy_next_upstream off;

      # Show which replica served each response, so you can match a
      # failure to the replica that did not have the state.
      add_header X-Upstream $upstream_addr always;
    }
  }
}

This minimal nginx config puts two local app replicas behind strict round-robin with no session affinity and no retries, and adds an X-Upstream header so each login, upload or checkout failure can be traced to the replica that served it.

Sessions: client-side cookies, shared stores and sticky sessions

Sessions are usually the first thing to break, because the default in most frameworks is an in-memory or local-file store. There are two replica-safe designs.

Stateless sessions put the session data in a cookie that the server signs (to detect tampering) or encrypts (to also hide the contents). Any replica can validate it, provided every replica uses the same secret. During key rotation, replicas must sign with the new key and still accept the old one until existing cookies expire. Otherwise a deploy logs out everyone whose cookie was issued by the old key. The costs are a size limit of roughly 4 KB per cookie and extra bytes on every request, including static assets on the same domain. Revocation is also hard: a signed cookie stays valid until it expires. A denylist of revoked session IDs fixes that, but the denylist has to be shared by all replicas, which brings back shared state.

Server-side sessions keep only an opaque ID in the cookie and store the data in a shared store such as Redis or a database table. Revocation is a delete. The cost is a network lookup per request and one more dependency to keep available.

Sticky sessions route each client back to the same replica, using either a load-balancer cookie or the client's source IP. Source-IP affinity is the weaker option because many users behind one NAT all land on the same replica. Either way, affinity hides the problem rather than fixing it:

• Restarts and scale-in log out every user pinned to the removed replica.
• Load skews toward replicas that hold long-lived clients.
• The local-state bugs you found are still there.

Affinity is a reasonable stopgap for a few weeks while you move sessions to a shared store. Avoid it when you autoscale aggressively or deploy often, because both remove replicas regularly, and avoid it as a permanent design.

Copyimport base64, hashlib, hmac, json, os, time

# Every replica loads the SAME keys from config, never per-process random values.
# Newest key first: sign with keys[0], accept any key in the list.
# Placeholders: real values come from your secret store.
KEYS = [os.environ.get("SESSION_KEY_NEW", "placeholder-new-key").encode(),
        os.environ.get("SESSION_KEY_OLD", "placeholder-old-key").encode()]
MAX_AGE = 3600  # seconds; also the worst-case revocation delay without a denylist

def _mac(key: bytes, payload: bytes) -> str:
    return hmac.new(key, payload, hashlib.sha256).hexdigest()

def sign_session(data: dict) -> str:
    # Signed, not encrypted: the client can read this, but cannot alter it.
    body = dict(data, iat=int(time.time()))
    payload = base64.urlsafe_b64encode(json.dumps(body).encode())
    cookie = payload.decode() + "." + _mac(KEYS[0], payload)
    if len(cookie) > 4000:  # browsers cap a cookie at roughly 4 KB
        raise ValueError("session too large for a cookie")
    return cookie

def load_session(cookie: str) -> dict | None:
    try:
        payload_s, sig = cookie.rsplit(".", 1)
    except ValueError:
        return None
    payload = payload_s.encode()
    # Accept old and new keys during rotation so a deploy does not log users out.
    if not any(hmac.compare_digest(_mac(k, payload), sig) for k in KEYS):
        return None
    try:
        data = json.loads(base64.urlsafe_b64decode(payload))
    except ValueError:
        return None
    if time.time() - data.get("iat", 0) > MAX_AGE:
        return None  # expiry is the only revocation unless a shared denylist is checked
    return data

A minimal HMAC-signed session cookie in standard-library Python that all replicas can validate with a shared secret, signing with the newest key while still accepting the old one during rotation.

Uploads: moving to shared file storage and direct-to-storage flows

A file written to one replica's disk does not exist for the others. The next request for that file may hit a replica that never saw it, and the file disappears whenever that container is replaced. Every user-generated file needs to live in storage that all replicas share. The two usual options trade off differently.

A network filesystem is faster to adopt. Object storage avoids the locking and caching problems that only show up once many replicas are under load.

Direct-to-storage uploads

If the storage supports pre-signed URLs, the app can issue a short-lived upload URL and the client sends bytes straight to storage. This keeps large request bodies off your replicas. The cost is that the app never sees the file before it is stored. Size checks, type checks and malware scanning become an asynchronous step that runs after the upload. Model each file with explicit states such as pending, accepted and rejected. Never serve a file in the pending state. Clean up any rejected objects.

Chunked uploads

Store upload progress in shared state, either a database row or the storage's native multipart session, and never in replica memory. Keep the upload ID, the parts received and the expected size there. Any replica can then accept the next chunk or finalize the upload after a retry or a deploy.

Copy-- Upload state lives in the shared database, not in replica memory,
-- so any replica can resume, finalize or reject an upload.
CREATE TABLE uploads (
  id            UUID PRIMARY KEY,
  owner_id      BIGINT NOT NULL,
  object_key    TEXT NOT NULL UNIQUE,      -- where the bytes live in shared storage
  storage_upload_id TEXT,                  -- native multipart session ID, if used
  status        TEXT NOT NULL DEFAULT 'pending'
                CHECK (status IN ('pending', 'accepted', 'rejected')),
  reject_reason TEXT,
  expires_at    TIMESTAMPTZ NOT NULL,      -- abandoned uploads are cleaned up after this
  created_at    TIMESTAMPTZ NOT NULL DEFAULT now()
);

-- One row per received chunk; a retried chunk overwrites its own row.
CREATE TABLE upload_parts (
  upload_id   UUID NOT NULL REFERENCES uploads(id) ON DELETE CASCADE,
  part_number INT  NOT NULL,
  etag        TEXT NOT NULL,               -- checksum/ID returned by storage for the part
  size_bytes  BIGINT NOT NULL,
  PRIMARY KEY (upload_id, part_number)
);

-- Resume: any replica can tell the client which parts are still missing.
SELECT part_number FROM upload_parts WHERE upload_id = $1 ORDER BY part_number;

-- Post-upload scan result: a conditional update so only one worker wins,
-- even if two replicas process the same completion event.
UPDATE uploads
   SET status = 'accepted'
 WHERE id = $1 AND status = 'pending';

-- Serve only accepted files; pending and rejected rows are never visible.
SELECT object_key FROM uploads WHERE id = $1 AND status = 'accepted';

-- Cleanup job: find rejected or expired uploads whose objects must be deleted.
SELECT id, object_key, storage_upload_id FROM uploads
 WHERE status = 'rejected'
    OR (status = 'pending' AND expires_at < now());

A PostgreSQL schema and queries that keep upload status and chunk progress in shared state, so any replica can resume, finalize, serve or clean up an upload.

Caches and background jobs: invalidation, schedulers and exactly-once concerns

Caches carry the same risk as uploads. Classify every cache. A per-process cache is fine when a stale read is harmless and a TTL caps how stale it can get. Feature flags refreshed every 30 seconds are a good example. A shared cache is needed when one replica's write must be visible on the next request, whichever replica serves it. For per-process caches, pick an invalidation method on purpose:

If you use broadcasts, add a TTL as well. A missed message then only causes temporary staleness.

Schedulers

In-process cron jobs and timers run once per replica. With four replicas, the nightly invoice job runs four times. There are two ways to fix this. You can enqueue the work and let workers consume it, so the queue decides who runs it. Or you can elect a leader or take a distributed lock. Locks are weaker than they look. A lock can expire while a long job is still running, and then a second runner starts. Clock drift, GC pauses and network partitions can also produce two holders at the same time.

Make jobs idempotent

Treat any single-runner mechanism as an optimization, not a guarantee. Give each job a deterministic key, such as invoice:2024-06:customer-812. Record that key in the same database transaction as the job's side effects, with a unique constraint on the key column. If a duplicate run happens, it hits the constraint and exits without repeating the work.

Deploys, health checks and mixed-version replicas

Once state lives outside the process, the load balancer's view of each replica matters more. A replica that starts fine but cannot reach Redis or the object store will accept traffic and fail every request that touches a session.

• Readiness checks verify shared dependencies. The readiness endpoint should do a cheap round-trip to the session store, cache and database, with a short timeout. Keep liveness separate and process-local. If liveness also checks shared dependencies, a single store blip makes the orchestrator restart every replica at the same moment.
• Drain before exit. On SIGTERM, fail readiness first. Wait long enough for the load balancer to stop routing to the replica, then finish in-flight requests and uploads, then exit. Set the orchestrator's grace period longer than your slowest legitimate request.
• Expect mixed versions. During a rolling deploy, old and new replicas read and write the same session and cache entries. If the new code renames a field or changes the serializer, old replicas will crash on data the new ones wrote. Add a format version to each payload, make readers tolerate unknown fields, and ship the reader change one release before the writer change.
• Decide the failure mode in advance. Each store is a new single point of failure and adds a network hop to every request. A cache outage should normally degrade the service: the app falls back to the database, with tight timeouts so requests don't pile up. A session store outage logs everyone out. Decide explicitly whether that means a read-only mode or a hard failure, and make the store's availability match the choice.

Rollout sequence, alternatives and impact on cost and engineering time

Rollout order

1. Externalize state on a single replica first. Ship the session, upload, cache and scheduler changes while only one instance is running. Any regression then comes from the code change and not from the topology.
2. Run two replicas in staging with affinity off. Repeat the reproduction tests from the audit. Two replicas are enough to expose local state.
3. Add production replicas behind the load balancer. If you added sticky sessions as a safety net, treat them as temporary and set a removal date.
4. Remove affinity. While it is enabled, it can hide replica-local bugs that only show up when an instance restarts.

Alternatives

Skip horizontal scaling if one machine handles peak load and short deploy downtime is acceptable. Also skip it if the database is the bottleneck, because each new replica adds connections to it.

Cost versus time saved

Illustrative estimate (assumed numbers, not a benchmark):

Assumed inputs:

• Upfront work: 120 engineer-hours
• New store operations: 2 h/month
• Replica-specific incidents avoided: 8 h/month
• Deploy coordination avoided: 3 h/month

Formula: break-even months = upfront ÷ (hours saved − ops hours)

Calculation: 120 ÷ (11 − 2) ≈ 13 months

This estimate leaves out the availability gained from having a second instance. Add that to your own figures when you plug them in.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to scaling monolith horizontally, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.