Scale a Monolith Horizontally: Stateless Sessions, Files and Cache

What does it mean to scale a monolith horizontally, and why does local state break it?

To scale a monolith horizontally, you run N identical copies of the same build behind a load balancer, and any copy can serve any request. You don't split the codebase into services. You find every place where one process quietly assumes it is the only one, and you move that state somewhere all instances can reach.

The usual offenders are sessions held in process memory, uploads written to local disk, caches that must agree across requests, and scheduled jobs that should run exactly once. With one instance these assumptions hold. Add a second instance and a user who logged in on A gets routed to B and appears logged out. A file uploaded to A returns a 404 from B. A nightly job runs twice.

CopyBEFORE: one stateful instance       AFTER: N stateless instances

 clients                             clients
    |                                   |
 [ app ]                          [ load balancer ]
  - sessions in memory              /      |      \
  - uploads on local disk       [app 1] [app 2] [app N]
  - in-process cache                \      |      /
  - cron in process            +----------+-----------+
                               | session store        |
                               | object storage       |
                               | shared cache         |
                               | job queue            |
                               +----------------------+

Teams make this change for three reasons:

• More throughput. You add instances instead of buying a bigger machine.
• Zero-downtime deploys. You replace instances one at a time while the others keep serving.
• Surviving instance loss. A crashed node costs capacity but not user state.

This approach fits monoliths limited by CPU or request handling. If a single database is the bottleneck, adding app instances only sends more load to it.

CopyBEFORE: one stateful instance       AFTER: N stateless instances

       +---------+                               +---------+
       | clients |                               | clients |
       +----+----+                               +----+----+
            |                                         |
            v                                         v
 +----------+----------+                      +---------------+
 |    app (single)     |                      | load balancer |
 |---------------------|                      +-------+-------+
 | sessions in memory  |                              |
 | uploads on local fs |                    +---------+---------+
 | in-process cache    |                    v         v         v
 | cron in process     |                +-------+ +-------+ +-------+
 +---------------------+                | app 1 | | app 2 | | app N |
                                        +---+---+ +---+---+ +---+---+
 a second copy breaks:                      |         |         |
 - user logged out on B                     +---------+---------+
 - upload 404s on B                                   |
 - nightly job runs twice                             v
                                    +-----------------+------------------+
                                    | session store   user sessions      |
                                    | object storage  uploaded files     |
                                    | shared cache    cached reads       |
                                    | job queue       run-once jobs      |
                                    +------------------------------------+

A single instance keeps sessions, uploads, cache and cron inside its own process, while N identical stateless instances behind a load balancer move that state into a shared session store, object storage, shared cache and job queue so any instance can serve any request.

Who should scale a monolith horizontally?

This approach suits teams that have a working monolith and need more capacity or availability than one instance provides, but have no other reason to split the codebase. Typical signs:

• The app tier is the bottleneck. CPU or worker saturation shows up on the app host during peaks, while the database still has headroom for connections, queries and I/O.
• Deploys need downtime. Restarting the single instance drops in-flight requests or requires a maintenance window.
• One machine is a single point of failure. A host failure or a bad kernel patch takes the whole product offline.
• Vertical scaling is running out. The next machine size up is expensive or does not exist.
• The team is small relative to the codebase. Running and versioning several services would cost more than the scaling problem itself.

It is a poor fit in three cases. If the database is already saturated, more app instances add load where there is no room for it, so fix queries, add read replicas or scale the database first. If one instance comfortably handles peak load and brief restarts are acceptable, the extra shared dependencies are not worth running yet. If the core of the product depends on large in-memory state or long-lived connections, read the section on avoiding full statelessness before starting.

How does horizontally scaling a stateless monolith work?

The mechanism has four parts.

A load balancer distributes requests. It sends each request to any healthy instance using a policy such as round-robin or least-connections. It decides which instances are healthy by polling a health or readiness endpoint on each one, and it stops routing to an instance that fails the check.

Every instance runs the same build and configuration. Instances differ only in identity, not behaviour. Configuration comes from the environment or a central config source, not from files edited on one host. This lets you add an instance by starting another copy and remove one by stopping it.

Request-scoped work stays in the process; everything else goes to shared backends. A typical authenticated request runs like this:

1. The load balancer forwards the request to, say, instance 2.
2. Instance 2 reads the opaque session ID from the cookie and loads the session from the shared store, or verifies a signed token.
3. It reads data from the database, using the shared cache for hot reads and its local cache only for data that tolerates staleness.
4. If the request involves a file, it reads or writes the object by key in object storage.
5. If the request triggers slow or scheduled work, it enqueues a job instead of running it in-process. A worker, possibly on another instance, picks it up.
6. The response goes back. Nothing instance 2 learned during the request is needed to serve the next one.

Deploys and failures become routine. For a rolling deploy, you start a new instance, wait for its readiness check to pass, then drain an old instance by removing it from the load balancer and letting in-flight requests finish before it stops. A crashed instance fails its health check and drops out of rotation, and its users' next requests land elsewhere with their sessions intact.

One shared resource needs attention as you add instances: database connections. Each instance keeps its own connection pool, so the total connections the database sees is roughly the instance count multiplied by the per-instance pool size. Size pools against the database's connection limit, or put a connection pooler in front of it, before scaling out.

How do you audit a monolith for hidden local state?

Before adding a second instance, inventory every place the process remembers something between requests. Run two instances behind a round-robin load balancer in staging. Bugs that appear only there are the state you missed.

Start with text searches, then confirm each hit by reading the code:

• Disk writes: grep -rnE "open\(.*['\"]w|writeFile|FileOutputStream|move_uploaded_file|/tmp/"
• Mutable globals: grep -rnE "static .*(Map|Dictionary|List)<|^[A-Z_]+ = \{\}|@lru_cache|memoize"
• Timers and schedulers: grep -rnE "setInterval|setTimeout|ScheduledExecutor|@Scheduled|Timer\(|cron"
• Locks: grep -rnE "synchronized|Mutex|threading.Lock|flock|\.lock\b"

Grep misses implicit state. Check these by hand:

• A multi-step flow that keeps a wizard step or an OAuth state value in memory.
• WebSocket or SSE connections, which stay bound to one instance.
• Feature flags loaded once at boot.
• IDs from in-process counters.

Copy#!/usr/bin/env bash
set -u

SRC="${1:-.}"
OUT="${2:-state-audit}"
mkdir -p "$OUT"

EXCLUDES=(--exclude-dir=.git --exclude-dir=node_modules --exclude-dir=vendor
          --exclude-dir=dist --exclude-dir=build --exclude-dir=target)

scan() {
  local name=$1 pattern=$2
  # grep exits 1 when nothing matches. That result is fine here, so ignore it.
  grep -rnE "${EXCLUDES[@]}" -e "$pattern" "$SRC" > "$OUT/$name.txt" || true
  printf '%-10s %5s hits -> %s\n' "$name" "$(wc -l < "$OUT/$name.txt")" "$OUT/$name.txt"
}

scan disk     "open\(.*['\"]w|writeFile|FileOutputStream|move_uploaded_file|/tmp/"
scan globals  'static .*(Map|Dictionary|List)<|^[A-Z_]+ = \{\}|@lru_cache|memoize'
scan timers   'setInterval|setTimeout|ScheduledExecutor|@Scheduled|Timer\(|cron'
scan locks    'synchronized|Mutex|threading\.Lock|flock|\.lock\b'
scan sessions 'MemoryStore|HttpSession|\$_SESSION|session\['
scan counters 'AtomicInteger|AtomicLong|itertools\.count|nextId|counter\+\+'

A bash script that runs the audit's grep patterns per state category, skips dependency and build directories, and writes each category's hits to its own file for manual review.

How should you move sessions out of process, and why are sticky sessions not enough?

Sessions are usually the first piece of state to move, because every authenticated request touches them. There are two viable designs.

A shared session store keeps session data in Redis or a database table, and the cookie carries only an opaque ID. Revocation is immediate: you delete the record. The costs are a network hop on every request and a new shared dependency. If the store is a single node, losing it logs everyone out, so it needs replication and failover.

Signed or encrypted client-side tokens put the data in the cookie itself. This removes the lookup, but it introduces three problems. You cannot revoke a token before it expires without adding a denylist, and that denylist is shared state again. Cookie size grows, and the cookie is sent on every request. Key rotation needs an overlap window in which both the old and new keys are accepted.

A typical store configuration sets these values:

• a key prefix such as sess:
• an idle TTL of 30 minutes, refreshed on activity
• an absolute lifetime of 12 hours
• cookies marked HttpOnly, Secure and SameSite=Lax
• a connection timeout short enough that a slow store fails fast

Keep the session small: user ID, authentication state and CSRF token. Leave out large objects and cached permissions that must take effect when revoked. Load those per request instead.

Sticky sessions only defer the problem. They hide state bugs until an instance is drained during a deploy or crashes, at which point its users lose their sessions. They also skew load, because long-lived clients stay pinned to one instance.

Copysession:
  store:
    backend: redis
    # More than one endpoint: a single node would log everyone out if it failed.
    endpoints:
      - redis-a.internal:6379
      - redis-b.internal:6379
      - redis-c.internal:6379
    key_prefix: "sess:"          # namespaces session keys; eases scans and cleanup
    connect_timeout_ms: 200      # a slow store fails fast instead of piling up requests
    command_timeout_ms: 100

  lifetime:
    idle_ttl: 30m                # expires after 30 minutes without activity
    refresh_on_activity: true    # each request pushes the idle expiry forward
    absolute_max: 12h            # hard cap, however active the user is

  cookie:
    name: sid                    # carries only an opaque random ID, never user data
    http_only: true              # page scripts cannot read it
    secure: true                 # sent only over HTTPS
    same_site: Lax               # not sent on most cross-site subrequests
    path: /

  # Store only what each request needs. Keep the record small.
  allowed_fields:
    - user_id
    - auth_state                 # e.g. authenticated, mfa_pending
    - csrf_token
  # Load these per request so a revocation takes effect on the next request:
  # roles and permissions, profile objects, cart contents, query results.

A framework-neutral session-store configuration: a replicated Redis backend with a key prefix and short timeouts, a 30-minute idle TTL and a 12-hour absolute cap, hardened cookie flags, and a minimal set of session fields.

How do you handle file uploads with object storage instead of local disk?

A file written to one instance's disk is invisible to the others, and it vanishes when that instance is replaced. Uploads have to go to object storage, and the database should record the object key (for example uploads/{tenant}/{uuid}), never a local path or a full provider URL.

You can stream uploads through the app, which keeps validation synchronous but ties up workers and bandwidth. The alternative is to issue pre-signed URLs so clients upload directly to storage. The direct-upload flow looks like this:

1. Client requests an upload. The app checks authorization, generates a key, and stores a pending record with the expected size and type.
2. The app returns a short-lived pre-signed PUT URL scoped to that key.
3. Client uploads, then calls confirm(key).
4. The app reads the object's metadata, checks size and type, marks the record uploaded, and enqueues a malware scan. The record becomes available only when the scan passes.

Direct upload moves validation into this asynchronous post-upload step. It also creates orphaned objects whenever a client never confirms, so add a lifecycle rule that expires them by prefix or age.

Derived files such as thumbnails and exports belong in object storage too. Local disk is scratch space only, and it must be cleaned up before the request or job ends.

Object storage is not a filesystem. It has no atomic rename and no file locking, and its latency is higher than local disk. Listing and consistency behavior varies by provider. Code that assumes POSIX semantics will break.

Copyimport uuid
import boto3

s3 = boto3.client("s3")
BUCKET = "example-app-uploads"
MAX_BYTES = 10 * 1024 * 1024
ALLOWED_TYPES = {"image/png", "image/jpeg", "application/pdf"}

def request_upload(user, tenant_id, content_type, size):
    # Authorize and reject obviously bad requests before issuing a URL
    if not user.can_upload(tenant_id) or content_type not in ALLOWED_TYPES or size > MAX_BYTES:
        raise PermissionError("upload not allowed")
    key = f"uploads/{tenant_id}/{uuid.uuid4()}"
    # Store the object key (not a path or full URL) with what we expect to receive
    db.insert_upload(key=key, status="pending", expected_size=size, expected_type=content_type)
    # Short-lived URL scoped to this one key; the client must send the same Content-Type
    url = s3.generate_presigned_url(
        "put_object",
        Params={"Bucket": BUCKET, "Key": key, "ContentType": content_type},
        ExpiresIn=300,
    )
    return {"key": key, "url": url}

def confirm(user, key):
    record = db.get_upload(key)
    if record is None or record.status != "pending" or not user.can_upload(record.tenant_id):
        raise PermissionError("unknown or already confirmed upload")
    # Trust storage metadata, not the client's claims
    head = s3.head_object(Bucket=BUCKET, Key=key)
    if head["ContentLength"] > MAX_BYTES or head["ContentType"] != record.expected_type:
        s3.delete_object(Bucket=BUCKET, Key=key)
        db.update_upload(key, status="rejected")
        return
    db.update_upload(key, status="uploaded", size=head["ContentLength"])
    # Record becomes "available" only after the async malware scan passes
    queue.enqueue("scan_upload", key=key)
    # Never-confirmed objects under uploads/ are removed by a bucket lifecycle rule

A Python sketch using boto3 that issues a short-lived pre-signed PUT URL for a generated object key, then confirms the upload by checking the stored object's metadata before queuing a malware scan; db, queue and user are application placeholders.

When is a per-instance cache fine, and when do caches and background jobs need shared coordination?

Caches are the one kind of local state you can often keep, as long as you are deliberate about what each layer holds.

Layer the cache by consistency needs

A per-instance in-memory cache works for data that never changes once written, such as compiled templates, feature metadata loaded at boot, or content-addressed lookups. It also works for data where brief staleness is acceptable, provided the TTL is short. The cost is that each instance can serve a value up to one full TTL old, and different instances will disagree during that window. Data that must be invalidated consistently, such as permissions, prices or account status, belongs in a shared cache. The alternative is to keep it local and broadcast invalidation messages so every instance evicts the key.

Shared caches introduce stampedes. When a hot key expires, every instance misses at once and hits the database together. Request coalescing, a short lock on recomputation, or jittered TTLs keep that miss from becoming an outage.

Stop running cron N times

A scheduler embedded in the monolith fires on every instance. Move that work onto a shared queue that workers consume. If the job must stay in-process, use leader election or a distributed lock so only one instance runs it.

Locks are not a guarantee. A lock can expire during a long job, clocks can skew, and a network partition can produce two leaders. The job itself must be idempotent. For a nightly invoice run, insert a row keyed on the customer ID and billing period into a table with a unique constraint before doing any work. If the insert fails, another run already claimed that period, so skip it.

When should you avoid full statelessness, and what are the alternatives?

Some workloads resist statelessness, and forcing it costs more than it saves. Long-lived WebSocket connections are bound to the process holding the socket. A large in-memory dataset, such as a pricing graph or search index, may take minutes to load and would be slow to fetch over the network on every request. Low-latency computation over local state, such as matching or game simulation, depends on that state staying in memory.

Each move toward statelessness adds a dependency to provision, a component in the on-call rotation, and code paths for timeouts and partial failure. Compare that cost with the alternative of splitting the monolith into microservices, which multiplies all three. Usually the cheaper option is to isolate the one stateful piece and keep the rest of the monolith stateless.

If you keep sticky sessions, schedule their removal and track it as a ticket with an owner. Otherwise new code will start depending on node-local state again.

How can a stateless monolith save engineering time?

The migration itself costs effort, but it removes several recurring costs.

• No service decomposition. You get more capacity and redundancy while keeping one codebase, one build, one deploy pipeline and in-process function calls. There are no inter-service API contracts to version, no distributed tracing needed to follow a single request, and no coordinated multi-service releases.
• No maintenance windows for deploys. Rolling deploys with draining mean releases can happen during working hours with no announcement, no off-hours staffing and no waiting for a quiet period. Teams tend to ship smaller changes when deploys are cheap, which makes each one easier to review and roll back.
• Instance failures stop being incidents. When a host dies, the load balancer routes around it and the replacement starts from the same image. Nobody has to recover files from its disk or work out which users lost their sessions.
• Replace instead of debug. A misbehaving instance can be terminated and replaced, because it holds nothing unique. Investigation can happen afterwards from logs and metrics rather than under pressure on a live host.
• Capacity changes become configuration. Handling a traffic peak means changing an instance count or autoscaling policy, not planning a migration to a larger machine.
• State bugs surface early. The two-replica harness described below turns intermittent, production-only problems such as random logouts or missing files into failures that reproduce in CI.

An illustrative estimate, with all inputs assumed: suppose a team deploys three times a week, and each deploy currently needs one engineer for an assumed one-hour off-hours maintenance window, plus an assumed four hours a month spent on incidents caused by the single host restarting or failing. That is roughly 12 hours a month of deploy supervision and 4 hours of incident work, about 16 engineer-hours a month. If rolling deploys reduce supervision to a few minutes of watching dashboards and host loss no longer pages anyone, most of that time comes back. Substitute your own deploy frequency and incident history; the result depends entirely on those inputs, and it should be weighed against the one-time migration effort and the ongoing cost of running the session store, cache and queue.

How do you verify statelessness and roll it out safely?

Verification checklist

Run at least two app replicas behind a load balancer that uses round-robin with stickiness explicitly disabled. A minimal setup is a compose file with two copies of the app container, plus a reverse proxy that lists both as upstreams. Point both at the same session store, object storage bucket and database. Two replicas are the minimum because one replica hides every local-state bug.

• Sessions: Log in, then stop the replica that served the login. The next request should still be authenticated.
• Files: Upload through one replica, then fetch the file through the other. Do the same after deleting and recreating a container, which proves nothing lives on its disk.
• Jobs: Leave both replicas running past a scheduled job's trigger time. Confirm the job ran exactly once, using its log or a side effect such as a row count.
• Caches: Update a record through one replica and read it through the other. Check that the staleness matches the TTL you designed for.

Rollout order

Migrate sessions first. That change is the smallest and lets you drop stickiness sooner. Move files next, then caches and jobs, which carry the subtlest coordination bugs.

For existing state, you have two options:

• Clean cutover: Accept a one-time logout, or freeze uploads while you copy files to the bucket.
• Dual-read period: Read from the new store and fall back to the old one.

Dual-read avoids user impact, but it adds fallback branches and per-instance edge cases, and the old path needs its own removal ticket.

Keep the two-replica harness in CI. It will catch regressions that reintroduce local state before production does.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to scale monolith horizontally, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.