What does it take to scale a monolith horizontally?
To scale a monolith horizontally, you run N identical copies of the same application behind a load balancer. Each replica is a full instance of the monolith, and any of them can serve any request. You do not need to split the codebase into services. The real work is finding every place where the process assumes it is the only copy running: sessions held in memory, files written to local disk, a cron loop inside the app, a counter cached in a global variable. Each of those assumptions breaks once a second replica exists.
Teams usually make this change for three reasons:
- Throughput. One process or host eventually runs out of CPU or connections.
- Availability. If a single instance crashes, every request fails.
- Zero-downtime deploys. With several replicas, you can replace them one at a time.
This checklist is for backend and platform engineers who own an existing stateful application and need it to run safely on several replicas.
A fully stateless app is a useful goal, but rarely the realistic one. The application still has state. You move it out of the process into shared stores that every replica can reach:
+----------------+
clients ----> | load balancer |
+-------+--------+
+------------+------------+
v v v
+---------+ +---------+ +---------+
|replica 1| |replica 2| |replica N|
+----+----+ +----+----+ +----+----+
+------------+------------+
+-----------+-------+-----+-------------+
v v v v
+----------+ +---------+ +-----------+ +----------+
| session/ | | object | | job queue | | database |
| cache | | storage | | | | | |
+----------+ +---------+ | worker | +----------+
+-----------+
+-----------------+
clients ------> | load balancer |
+--------+--------+
|
+-------------------+-------------------+
| | |
v v v
+-----------+ +-----------+ +-----------+
| replica 1 | | replica 2 | ... | replica N |
+-----+-----+ +-----+-----+ +-----+-----+
| | |
+-------------------+-------------------+
|
+-----------------+--------+--------+-----------------+
| | | |
v v v v
+-----------+ +-----------+ +-----------+ +-----------+
| session / | | object | | job queue | | database |
| cache | | storage | +-----+-----+ +-----+-----+
+-----------+ +-----------+ | ^
v |
+-----------+ |
| worker |-----------+
+-----------+
N identical monolith replicas behind a load balancer, with sessions, cache, files, background jobs and data moved out of the process into shared stores that every replica can reach.
Who should use this approach?
Horizontal scaling of a monolith fits teams in these situations:
- You own a working monolith and don't want a rewrite. The codebase is valuable, the domain logic is tangled in ways that make service boundaries unclear, and splitting it would take months before it delivers anything. Replicas let you keep one codebase and one build.
- You have outgrown one instance, or you can't tolerate it failing. CPU, memory or connection limits on a single process are the constraint. Or a single crash or deploy takes the whole product offline.
- Your database still has headroom. The app tier is the bottleneck, not the primary database. If the database is already saturated, see the section on when to avoid horizontal scaling.
- You can change application code. Moving sessions, files, caches and scheduled jobs out of the process requires code changes. Infrastructure changes alone won't do it.
Within those teams, the work usually falls to two roles. Backend engineers audit and fix in-process state, locking and job scheduling in the application. Platform or SRE engineers set up the load balancer, health checks, shared stores, connection limits and the rolling-deploy pipeline. The checklist below is split so each group can see which items it owns.
If none of these apply, because the app runs comfortably on one machine and occasional restart downtime is acceptable, you probably don't need this yet.
How does horizontal scaling of a monolith work?
The mechanism has four parts.
-
Identical replicas. Every replica runs the same build with the same configuration. Nothing in a replica's local memory or disk should be needed by a later request. Because of that, any replica can serve any request, and you can add or remove replicas without coordinating with the others.
-
A load balancer in front. Clients connect to the load balancer, not to replicas. For each request or connection, it picks a healthy replica using a balancing policy such as round-robin or least connections. It learns which replicas are healthy from readiness checks: a replica gets traffic only after its readiness endpoint succeeds, and stops getting traffic when that endpoint fails.
-
State in shared tiers. Each kind of state lives where every replica can reach it:
- Sessions go in a shared store or a signed token.
- Uploaded and generated files go in object storage.
- Cached values live in a shared cache or carry explicit TTLs and invalidation.
- Scheduled and background work goes on a queue consumed by workers.
- Durable data and cross-replica correctness rely on the database: constraints, row locks, version columns and idempotency keys.
-
Lifecycle through health checks and draining. To scale out, you start another replica. It warms up, passes readiness, and the load balancer begins sending it traffic. To scale in or deploy, a replica fails readiness, the balancer stops routing to it, it finishes in-flight requests, and it exits.
Here is one request end to end. A logged-in user uploads a file. The load balancer sends the request to replica 2. Replica 2 loads the session from the shared store, writes the file to object storage, and inserts a row in the database. It then enqueues a thumbnail job and returns. The user's next request lands on replica 1, which reads the same session and the same database row and can serve the file. A worker, not a web replica, picks up the thumbnail job. No step depends on which replica handled the previous one.
The rest of this checklist covers getting each part right: finding state that still lives in the process, moving it, keeping jobs and writes correct, and configuring the balancer, database and deploys.
How do you find hidden in-process state before adding replicas?
Before adding replicas, list every place the process remembers something between requests. Anything held in memory or on local disk is visible to only one replica, so it has to move to a shared tier or become safe to lose.
| In-process state | Symptom with two replicas | Typical fix |
|---|---|---|
| HTTP sessions | Random logouts, lost carts | Shared session store or signed tokens |
| In-memory caches | Stale reads after writes | Shared cache or explicit invalidation |
| Module-level singletons | Counters and registries diverge | Move data to database or cache |
| Rate-limit counters | Effective limit multiplied by replica count | Shared counter store |
| Feature-flag snapshots | Users flip between variants | Shared flag source with refresh |
| WebSocket connection maps | Messages miss users on other replicas | Pub/sub fan-out between replicas |
| Local temp files and uploads | 404s on follow-up requests | Object storage |
| In-process cron | Jobs run once per replica | Leader election or external scheduler |
To confirm the list, run two replicas in staging behind a round-robin proxy with two upstreams, as in the config example below. Round-robin sends consecutive requests to different processes, so state bugs show up on the second click instead of under rare production timing. Go through log in, upload, paginate, hit a rate limit, open a WebSocket, and wait for a scheduled job. Record every failure.
This audit takes hours. Finding the same bugs in production costs more because they look like intermittent, user-specific glitches that don't reproduce on a single developer instance. Each one has to be traced back to replica affinity before anyone can fix it.
events {}
http {
upstream app_replicas {
# No ip_hash or sticky option on purpose: affinity would hide state bugs.
server 10.0.0.11:8080;
server 10.0.0.12:8080;
}
server {
listen 80;
server_name staging.example.internal;
location / {
proxy_pass
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Shows which replica served each response in the access log.
add_header X-Upstream $upstream_addr always;
}
# WebSocket endpoint: same upstream pool, so a client on one replica
# will miss messages published from the other unless fan-out exists.
location /ws {
proxy_pass
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
}
A minimal staging nginx config that round-robins requests across two monolith replicas with no session affinity. An X-Upstream response header shows which replica served each request, so state bugs can be traced to a specific process.
How should you handle sessions, files and caches across replicas?
Sessions are usually the first thing to move, and the choice determines how logout and failures behave.
| Option | Logout and revocation | Session size | Failure dependency | Load evenness |
|---|---|---|---|---|
| Shared store (Redis or database) | Immediate: delete the key | Unbounded, server-side | Store outage logs everyone out or blocks requests | Even, any replica serves any user |
| Signed client-side token | Hard: valid until expiry unless you add a denylist, which brings back shared state | Limited by cookie and header size, sent on every request | None at read time | Even |
| Sticky sessions | Local only, lost when the replica dies | Unbounded, in memory | Losing a replica drops its users' sessions | Uneven: long-lived clients pin to hot replicas |
Treat sticky sessions as a stopgap while you migrate. They hide state rather than remove it, and they turn every deploy into a forced logout for some users.
Files. Uploads, generated reports and disk caches written to local disk exist on only one replica. Object storage is the safer target: every replica reads and writes it the same way, and it carries no filesystem semantics you might accidentally rely on. A shared network volume looks like a drop-in replacement, but it has costs. Consistency and caching across clients can be weaker than on a local disk. File locking may be unsupported or unreliable. UID and permission mismatches between containers cause write failures. The file server also becomes a single point of failure for every replica.
Caches. N local caches mean N possibly different answers. Choose per data type based on how much staleness it tolerates. Use a shared cache when replicas must agree, such as permissions or prices. Use short TTLs when a few seconds of drift is acceptable. Use pub/sub invalidation when you need fast convergence but still want local read speed. Invalidation messages can be lost, so keep a TTL underneath as a backstop.
How do you keep scheduled jobs and concurrent writes correct on many replicas?
An in-app scheduler starts on every replica, so with four replicas the nightly invoice job runs four times. A synchronized block or language-level mutex only coordinates threads inside one process. Across replicas it protects nothing.
Move scheduling out of the request path
- Separate worker deployment: run the scheduler as its own deployment with exactly one replica, and keep web replicas free of cron.
- Job queue: the scheduler only enqueues work, and any worker can consume it. This also gives you retries.
- Leader election or an advisory lock: each replica tries to take a lock (in PostgreSQL,
pg_try_advisory_lock(<job id>)), and only the replica that gets it runs the job.
None of these guarantee exactly-once execution. A lease can expire while a slow job is still running, and a network partition can leave two nodes that each believe they hold the lock. Every job must therefore be safe to run twice.
Replace in-process locks with database guarantees
- Unique constraints turn "check, then insert" races into a single atomic failure that you can handle.
- Row locks (
SELECT ... FOR UPDATE) serialize read-modify-write on one row for the length of a transaction. - Optimistic concurrency: add a
versioncolumn, writeUPDATE ... SET ..., version = version + 1 WHERE id = ? AND version = ?, and retry when zero rows are affected. - Idempotency keys for payments and counters: store each client-supplied key in a table with a unique constraint, in the same transaction as the side effect. A retried request then hits the constraint instead of charging the customer twice.
-- PostgreSQL. 1) Advisory lock: only one replica runs the nightly invoice job.
-- The _xact_ variant releases at COMMIT/ROLLBACK, so a pooled connection
-- cannot keep holding it after the job ends.
BEGIN;
SELECT pg_try_advisory_xact_lock(42001) AS got_lock; -- 42001 = invoice job id
-- If got_lock is false, ROLLBACK and exit: another replica owns this run.
-- If true, do the work here. It must still be idempotent (see below).
COMMIT;
-- 2) Optimistic concurrency: the write succeeds only if nobody changed the row
-- since we read version 7. Zero rows affected means re-read and retry.
UPDATE accounts
SET balance_cents = balance_cents - 2500,
version = version + 1
WHERE id = 1234
AND version = 7;
-- 3) Idempotency keys: the unique constraint makes a retried request a no-op.
CREATE TABLE idempotency_keys (
idem_key text PRIMARY KEY, -- client-supplied, e.g. a UUID
response jsonb, -- result to replay on retry
created_at timestamptz NOT NULL DEFAULT now()
);
BEGIN;
-- Claim the key. If RETURNING yields no row, this key was already processed:
-- ROLLBACK and return the stored response instead of charging again.
INSERT INTO idempotency_keys (idem_key)
VALUES ('8f14e45f-ceea-467f-a0e6-2c3b1f0a9d21')
ON CONFLICT (idem_key) DO NOTHING
RETURNING idem_key;
-- Side effect in the SAME transaction, so key and charge commit or fail together.
INSERT INTO payments (account_id, amount_cents, idem_key)
VALUES (1234, 2500, '8f14e45f-ceea-467f-a0e6-2c3b1f0a9d21');
UPDATE idempotency_keys SET response = '{"status":"charged"}'
WHERE idem_key = '8f14e45f-ceea-467f-a0e6-2c3b1f0a9d21';
COMMIT;
PostgreSQL examples of a transaction-scoped advisory lock for a scheduled job, an optimistic update guarded by a version column, and an idempotency-key table whose unique constraint stops a retried payment from being applied twice.
Configuring the load balancer, the database and rolling deploys
Load balancer and shutdown
Expose two health endpoints. Liveness answers "is this process stuck?" and should not check dependencies. If it does, a brief database outage fails liveness everywhere and the orchestrator restarts every replica at once. Readiness answers "should this replica receive traffic right now?" It fails during warm-up and during shutdown.
On SIGTERM, follow this order:
- Mark readiness as failing.
- Wait a few probe intervals so the load balancer stops routing to this replica.
- Stop accepting new connections.
- Let in-flight requests finish, with a deadline shorter than the orchestrator's kill timeout.
- Close connection pools and exit.
If you skip the wait in step 2, the balancer keeps sending requests to a socket that is closing, and clients see errors on every deploy. Set the load balancer's idle timeout longer than the app's keep-alive timeout. Otherwise the balancer can reuse a connection the app has already closed.
WebSocket and SSE connections stay on the replica that accepted them. After a scale-out, new replicas sit idle while old ones stay saturated. Cap connection lifetime and have clients reconnect with jitter.
The database becomes the bottleneck
Every replica opens its own connection pool. Illustrative estimate: assume 12 replicas, a pool of 20 per replica, and a database limit of 200 connections. You need 240 before counting migrations, admin sessions, or surge replicas during a deploy. Shrink the per-replica pools or put a connection pooler in front of the database.
Also review read-modify-write code that assumed a single writer. With many replicas it now races, or it holds locks long enough to cause contention and deadlocks.
Rolling deploys
During a rolling deploy, old and new code run against the same schema. Make schema changes in three steps, each in a separate release:
- Expand: add the new column as nullable.
- Migrate: backfill it and dual-write to old and new columns.
- Contract: drop the old column once no running code reads it.
import http from "node:http";
import { Pool } from "pg";
const pool = new Pool({ max: 10 }); // keep small: max x replicas must fit the DB limit
let ready = false;
const server = http.createServer(async (req, res) => {
if (req.url === "/livez") return res.end("ok\n"); // no dependency checks
if (req.url === "/readyz") {
res.statusCode = ready ? 200 : 503; // fails during warm-up and shutdown
return res.end();
}
const { rows } = await pool.query("SELECT now()");
res.end(JSON.stringify(rows[0]));
});
server.keepAliveTimeout = 65_000; // keep below the load balancer idle timeout
server.listen(8080, () => { ready = true; });
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
process.once("SIGTERM", async () => {
ready = false; // 1. readiness starts failing
await sleep(10_000); // 2. a few probe intervals so the LB stops routing here
// 3 + 4. stop accepting connections, let in-flight requests finish
const drained = new Promise<void>((r) => server.close(() => r()));
server.closeIdleConnections(); // drop idle keep-alive sockets now
const deadline = sleep(15_000); // shorter than the orchestrator's kill timeout
await Promise.race([drained, deadline]);
await pool.end(); // 5. close the pool, then exit
process.exit(0);
});
A Node.js server that keeps liveness and readiness separate and handles SIGTERM in order: fail readiness, wait for the balancer, drain under a deadline, then close the pool.
When should you avoid horizontal scaling, and what are the alternatives?
Making every replica interchangeable is expensive. Skip it when it doesn't solve your actual problem:
- The database is already the bottleneck. More app replicas mean more connections and more queries against the same primary. Throughput stays flat, and contention may get worse.
- The load fits on a larger machine. One bigger instance avoids every distributed-state problem covered earlier.
- You need availability, not throughput. Surviving a node failure takes a second instance, not a fleet.
| Option | Solves | Leaves unsolved |
|---|---|---|
| Vertical scaling | Throughput up to one machine's limit | Single point of failure, restart downtime |
| Active/passive pair | Node failure | Failover lag, idle capacity, split-brain risk |
| Sticky sessions only | Session affinity with little code change | Lost sessions when a replica dies, uneven load |
| Extract the hot path | Scaling one endpoint independently | A second deployable to own and operate |
Be honest about the cost. A full stateless migration adds a shared session store, a lock or leader-election mechanism, and object storage. Each one must itself be highly available, or you have just moved the single point of failure somewhere else. Expand/migrate/contract schema changes turn one refactor into several releases. Switching file handling to object storage touches every upload and download path.
The return comes later. You get fewer incidents caused by state living on one box, and deploys become routine because any replica can be drained and replaced. If you expect to grow past one machine, that trade usually pays off. If you don't, it usually doesn't.
How can horizontal scaling save engineering time?
The migration costs time up front. These are the places it gives time back:
- No service rewrite. You keep one codebase, one build, one test suite and one deploy pipeline. Splitting a monolith into services also means defining APIs between them, versioning those APIs, handling distributed transactions and running several pipelines. Adding replicas avoids all of that. The changes are limited to state handling, which is a smaller and more mechanical set of fixes.
- Deploys stop needing coordination. With one instance, every deploy is a restart, so teams batch changes and schedule maintenance windows. With drained rolling deploys, a release replaces replicas one at a time with no user-facing outage. Teams can ship smaller changes more often, and smaller changes are faster to review and to roll back.
- Fewer emergency responses to single-host failures. When one replica crashes or its host is lost, the load balancer routes around it and the orchestrator replaces it. The failure becomes something to investigate during working hours rather than an outage someone has to fix at night.
- Capacity becomes a configuration change. Once replicas are interchangeable, handling more load means raising the replica count or an autoscaling limit, within the database's connection and query budget. It no longer means a migration to a larger machine with planned downtime.
- State bugs surface in staging, not production. Running two or more replicas in every pre-production environment, as described in the audit and verification sections, turns intermittent production glitches into failures you can reproduce on the second request. Reproducible bugs take far less time to diagnose than ones tied to which process happened to serve a request.
- Correctness lives in one place. Moving locking and deduplication into database constraints, version columns and idempotency keys replaces ad-hoc in-process guards. You end up with fewer duplicate-charge or double-run incidents to clean up by hand, and the rules are visible in the schema rather than scattered through code.
Whether the time saved outweighs the migration effort depends on how often you deploy, how often single-instance failures hurt you, and whether you expect to outgrow one machine. If you rarely deploy and one machine is enough, the payback is small. If deploys are frequent and each one causes downtime or a forced logout, the savings start with the first rolling deploy.
What should you verify before running multiple replicas in production?
Run every pre-production environment with at least two replicas. Most state bugs stay hidden when one replica handles everything, so single-replica staging tells you very little. Then work through these checks:
- Kill a replica mid-request. Terminate it while it is serving traffic. Clients should see a retry or a clean error, not a corrupted write or a stuck transaction. This shows whether your graceful shutdown and retry logic actually work.
- Test logout and revocation across replicas. Log in through one replica, log out or revoke a token through another, and then reuse the old credential against each replica. A per-replica cache of sessions or permissions will let it through.
- Upload on one replica, read from another. Pin the requests if you need to. If the read fails, something is still writing to local disk.
- Confirm each scheduled job runs exactly once. Check job logs or side effects, such as emails sent or rows created, while several replicas are running. Repeat the check after a replica restarts during the job's window.
- Check total database connections at peak replica count. Multiply the pool size by the maximum number of replicas the autoscaler allows, add connections from deploy overlap, and compare the result with the database's limit.
- Run a rolling deploy with a schema change. Old and new code will run against the same schema at the same time. This is where non-backward-compatible migrations fail.
- Restart a replica that holds long-lived connections. WebSocket or streaming clients should reconnect, and load should spread back across replicas instead of staying piled on whichever replicas survived.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to scale monolith horizontally, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.