Blue-Green Deployment for a Monolith: Keeping the Database Compatible

What blue-green deployment means for a monolith with a shared database

Blue-green deployment runs two complete copies of your application side by side. Blue serves production traffic. Green runs the new release. You deploy to green, verify it with smoke tests or internal traffic, and then move the router or load balancer from blue to green. If green misbehaves, you move traffic back. Both stacks are already running, so the switch and the rollback take seconds. You don't rebuild artifacts or wait for a redeploy under pressure.

The textbook diagram shows two fully isolated environments. A monolith rarely works that way. Blue and green usually point at one database, and often one cache and one message queue as well. Copying a production database for each release, and keeping both copies consistent, is rarely practical. So both versions read and write the same schema during the traffic switch and for the whole rollback window after it.

That leads to the core rule: any schema in use must work with version N and version N+1 at the same time. The order matters. You usually migrate before green takes traffic, while blue is still serving every request. A migration that drops or renames something blue depends on breaks production before you switch anything.

There are two costs to plan for:

• Doubled compute. You need enough capacity for two full stacks, at least during each release.
• Shared blast radius. Any dependency you don't duplicate stays a single point of failure. A bad migration or a cache poisoned by the new release hits blue and green together, and moving traffic back won't undo it.

Copy                 +-------------------------------+
                 |    Router / Load Balancer     |
                 +-------------------------------+
                     |                       :
        live traffic |                       :  after switch
                     v                       v  (rollback: flip back)
      +-------------------------+ +-------------------------+
      |    BLUE  - version N    | |   GREEN - version N+1   |
      |  serves production now  | |  smoke tests, then live |
      +-------------------------+ +-------------------------+
      |<-------- two full stacks = doubled compute -------->|
                   |                           |
     --------------+---------------------------+--------------
          |                      |                      |
          v                      v                      v
     +---------+      +---------------------+      +---------+
     |  Cache  |      |      Database       |      |  Queue  |
     | shared  |      |  ONE schema, must   |      | shared  |
     |         |      |  work for N and N+1 |      |         |
     +---------+      +---------------------+      +---------+
                                 ^
                                 |
                 migrate here first, while blue serves all traffic

  Shared blast radius: a bad migration or a poisoned cache hits BLUE and
  GREEN together. Switching traffic back will not undo it.

Blue and green are separate application stacks behind one router, but both read and write the same database, cache and queue, so the shared schema must work for version N and N+1 at once.

Which schema changes are safe and which need a multi-step plan

The test for any change is simple: after the migration runs, can the version currently serving traffic keep working, and can it keep working after a rollback once the new version has written data? Both releases share one schema during the cutover window, so the schema has to suit both.

Additive changes are safe because old code never references what it doesn't know about. Destructive or restrictive changes remove something old code depends on, or reject writes it still makes.

The ORM can break this rule. A mapper that runs SELECT * and fails on unexpected columns turns a harmless new column into an outage. ORMs that read table metadata at startup often build INSERTs that list every column they know about. When a column is dropped, every write from the old release fails until it restarts. Before relying on blue-green for schema work, find out how your ORM behaves in both cases.

If a change can't be split so that each step is backward compatible, blue-green won't protect you. Plan a maintenance window for it.

How expand-and-contract works: renaming a column across releases

Expand-and-contract, also called parallel change, splits a breaking change into steps that each leave the schema readable and writable by both the old and new release. Take renaming orders.cust_ref to orders.customer_id:

1. Expand. Add customer_id as a nullable column with no default. Blue still runs and ignores it.
2. Dual-write. Release a version that writes both columns on every insert and update but still reads cust_ref. If you roll back to the previous release, nothing breaks.
3. Backfill. Copy cust_ref into customer_id in batches keyed by primary key, for example 5,000 rows per transaction. Small batches keep locks short and replication lag low.
4. Switch reads. Release a version that reads customer_id and still writes both. The previous release still finds a current cust_ref.
5. Stop writing the old column. Do this only after you have ruled out rolling back to step 4.
6. Contract. Drop cust_ref after the rollback window closes.

Step 2 can be implemented in the application or in the database:

Application dual-writes work when you control every writer. Triggers are safer when you don't, but give the trigger a clear name and track its removal as its own task.

Copy# Release 1 (expand), run as a migration first:
#   ALTER TABLE orders ADD COLUMN customer_id bigint;  -- nullable, no default
# Assumes a DB-API driver such as psycopg (%s placeholders).

BATCH_SIZE = 5000

def insert_order(cur, order_id, cust_ref, total):
    # Release 2+: write both columns so the old and new release can read the row.
    cur.execute(
        "INSERT INTO orders (id, cust_ref, customer_id, total) VALUES (%s, %s, %s, %s)",
        (order_id, cust_ref, cust_ref, total),
    )

def update_customer(cur, order_id, customer_id):
    # Every write path must set both columns. A path that misses one causes drift.
    cur.execute(
        "UPDATE orders SET cust_ref = %s, customer_id = %s WHERE id = %s",
        (customer_id, customer_id, order_id),
    )

def backfill(conn):
    # Walk the primary key in fixed ranges. Each batch is a short transaction.
    with conn.cursor() as cur:
        cur.execute("SELECT min(id), max(id) FROM orders")
        lo, hi = cur.fetchone()
    if lo is None:
        return  # empty table, nothing to copy
    start = lo
    while start <= hi:
        with conn.cursor() as cur:
            # IS NULL skips rows already written by the dual-write release.
            cur.execute(
                "UPDATE orders SET customer_id = cust_ref "
                "WHERE id >= %s AND id < %s AND customer_id IS NULL",
                (start, start + BATCH_SIZE),
            )
        conn.commit()  # release row locks before starting the next batch
        start += BATCH_SIZE

Application-side dual-writes for the cust_ref to customer_id rename, followed by a backfill that copies values in primary-key batches of 5,000 rows and commits after each batch.

Implementing migrations safely on a live shared database

Run migrations as their own pipeline step

Treat schema changes as separate deployable units, not something the application runs on boot. The expand migration ships and completes before green is started, so both blue and green see a schema they can work with. The contract migration (dropping the old column, trigger or index) goes out as a separate change once blue has been retired and the rollback window has closed. If the contract step ships in the same release as the code that stops using the old column, rolling back to blue breaks.

Keep locks short and fail fast

On a large table, many ALTER TABLE forms take a strong lock. Even a fast DDL statement can cause an outage: while it waits for that lock, queries arriving after it queue behind it. In PostgreSQL, set lock_timeout and statement_timeout for the migration session so a blocked migration aborts within seconds and can be retried, instead of blocking traffic. Build indexes with CREATE INDEX CONCURRENTLY. It cannot run inside a transaction, and if it fails it leaves an invalid index that you must drop and rebuild.

Throttle backfills

Backfill in small batches keyed by primary key, committing each batch and pausing between them. Large single updates create long transactions, cause table bloat, and generate write-ahead or binary logs faster than replicas can apply them. Watch replica lag and disk I/O, and slow down when either rises. The trade-off is a slower backfill in exchange for stable latency for users.

Enforce it in review

Add a CI check that flags destructive DDL such as DROP, RENAME, or type changes when it appears in the same change as application code. These changes should require an explicit sign-off before they merge.

Copy#!/usr/bin/env bash
# Expand migration for PostgreSQL. Runs as its own pipeline step, before green starts.
set -euo pipefail
DB="postgresql://migrator@db-primary.internal:5432/shop"

# Fail fast: abort if a lock isn't granted within 5s, so traffic doesn't queue behind us.
export PGOPTIONS="-c lock_timeout=5s -c statement_timeout=30s"

# Nullable column with no default: catalog-only change, so the lock is held only briefly.
psql "$DB" -v ON_ERROR_STOP=1 -c "ALTER TABLE orders ADD COLUMN IF NOT EXISTS status_code smallint;"

# Build the index without blocking writes. psql -c runs it outside a transaction block, which it requires.
# Lift statement_timeout because the build can take a long time. On failure, check pg_index.indisvalid,
# then drop the invalid index and rerun. IF NOT EXISTS would silently skip an invalid leftover.
PGOPTIONS="-c lock_timeout=5s -c statement_timeout=0" \
  psql "$DB" -v ON_ERROR_STOP=1 -c "CREATE INDEX CONCURRENTLY IF NOT EXISTS orders_status_code_idx ON orders (status_code);"

# Batched backfill by primary-key range. Each psql call autocommits, so transactions stay short.
max_id=$(psql "$DB" -At -c "SELECT coalesce(max(id), 0) FROM orders;")
batch=5000
last=0
while [ "$last" -lt "$max_id" ]; do
  psql "$DB" -v ON_ERROR_STOP=1 -c "
    UPDATE orders
       SET status_code = CASE status WHEN 'pending' THEN 1 WHEN 'paid' THEN 2 ELSE 0 END
     WHERE id > $last AND id <= $((last + batch)) AND status_code IS NULL;"
  last=$((last + batch))

  # Throttle on replica lag (seconds) reported by the primary. Back off hard when replicas fall behind.
  lag=$(psql "$DB" -At -c "SELECT coalesce(max(extract(epoch FROM replay_lag)), 0)::int FROM pg_stat_replication;")
  if [ "$lag" -gt 5 ]; then sleep 10; else sleep 0.2; fi
done
# Rows that blue writes after their range is backfilled stay NULL here. Cover them with a dual-write
# trigger or a final catch-up pass before green relies on the column.
# The contract step (dropping the old 'status' column) ships later as a separate change.

This script runs a PostgreSQL expand migration as a standalone pipeline step: lock and statement timeouts make it fail fast, CREATE INDEX CONCURRENTLY builds the index without blocking writes, and the backfill runs in primary-key batches and backs off when replica lag rises.

Traffic switch, non-schema shared state and rollback windows

Flipping the load balancer does not end the mixed-version period. Requests already in flight finish on blue. Sessions created by blue are read by green. Background workers and queue consumers from both colors may pull from the same queue, so a job that green enqueues can be handled by a blue worker. Message bodies and cache entries need the same compatibility as the schema: either version must be able to read what the other writes, or the keys must be versioned so the two never collide. Scheduled tasks such as billing runs or cleanup jobs should be enabled on exactly one color. If both colors run them, they run twice.

Rollback is not symmetric. You can switch back to blue only while blue can still read everything green has written. Define a rollback window, a set period after the switch during which reverting stays safe, and do not run any contract step until it has closed. Some changes cannot be reversed by switching traffic back. A new serialization format, values encrypted under a new key, or a backfill that rewrites existing rows are examples. Treat these as one-way doors that need explicit sign-off.

Pre-switch checklist

• Confirm that the migrations have been applied and that the schema matches what both versions expect.
• Run smoke tests against green using production data.
• Drain blue's workers, or confirm they can safely process green's messages.
• Choose between a gradual switch and an atomic one.
• Watch error rates and slow queries during the switch and after it.
• Keep blue warm and scaled until the rollback window closes.

Who should use it, when to avoid it, and the alternatives

Blue-green pays off most for a monolith that ships several times a week and needs a rollback measured in seconds. With one deployable unit, every release carries every team's changes, so the ability to send traffic back to blue without a redeploy is worth the cost of the second environment. Teams that release monthly, or that can tolerate a few minutes of rebuild during an incident, get less out of it.

When it is the wrong tool

Blue-green assumes both versions can read and write the same data at the same time. That assumption fails when:

• The data model is being rewritten, such as splitting one table into a different aggregate structure. The compatibility shims can cost more than the change.
• A value's meaning changes while its shape stays the same. If amount switches from cents to dollars, blue and green will silently corrupt each other's data. No schema check will catch this.
• Making the change backward compatible would take many releases. In that case, a planned outage is often cheaper and safer.

Alternatives

Only the maintenance window avoids the compatibility discipline, because nothing old is running while the schema changes. Its costs are downtime and a rollback that may mean restoring data.

Impact on development cost and engineering time

Making every schema change compatible across two live versions moves cost from deploy night into ordinary sprint work. A column rename that would be one migration inside a maintenance window becomes three or four releases: add the new column, dual-write and backfill, switch reads, then drop the old column. Each step waits until the previous one is live on both colors, so a single change can span days or weeks.

The contract step is the one teams forget. Without a ticket or a CI check that fails when a deprecated column outlives its deadline, you accumulate dead columns and dual-write code that every later developer has to read around. Throttled backfills run longer than a single bulk UPDATE, which is intentional, but someone still has to monitor them. During the switch you also run two full environments, which roughly doubles compute for that window.

The savings come from three places:

• Safer rollbacks. Rolling back means switching traffic, not restoring a schema, so fewer rollbacks fail halfway.
• No downtime coordination. You no longer schedule off-hours windows with support and on-call staff.
• Reuse. After the first few migrations, the steps become templates, and automated DDL checks catch unsafe changes before review.

Illustrative estimate (assumed numbers, not a benchmark)

• Extra cost per change = extra releases × overhead per release. Assuming 3 × 1.5 h, that is 4.5 h.
• Window cost = people × window hours + planning. Assuming 6 × 3 h + 4 h, that is 22 h.
• If one window would have carried 4 changes, compatibility costs 18 h against 22 h.

The two approaches are close at first. The balance shifts toward blue-green as templates reduce the per-release overhead.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to blue green deployment monolith, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.