Monolith Web and Worker Processes: Isolating Background Work

What splitting web and worker process types means, and why background work hurts web processes

Splitting web and worker process types means running the same codebase in two roles. You keep one repository and produce one build artifact, such as a container image or release bundle. That artifact starts with different entrypoints: one boots the HTTP server, and another boots a loop that pulls jobs from a queue. Some platforms call the second role a worker dyno. The code, dependencies and configuration are shared. Only the startup command and the resources assigned to each role differ.

The motivation is contention. When jobs run inside web processes, through in-process thread pools or "fire and forget" tasks spawned from request handlers, they compete with requests for the same finite resources:

• CPU. Report generation or image resizing can saturate cores, so request handling waits for scheduler time.
• Threads and event loops. Long jobs occupy workers that would otherwise accept connections, which increases queueing.
• Database connection pools. A batch job holding several connections leaves requests blocked on pool checkout.
• Memory. A job that loads a large dataset can push the process past its limit. The resulting OOM kill terminates every in-flight request on that instance along with the job.

Median latency often looks fine while p99 degrades, because only requests that happen to coincide with heavy jobs suffer. That makes the problem easy to miss on dashboards.

Teams that benefit most run a monolith with meaningful background load: imports, emails, webhooks and scheduled batch work. This is usually the cheapest isolation step available. It needs no service boundary, no new repository and no API contract. Reverting it means changing a start command.

Copy           +--------------------------------------------------+
           | ONE REPOSITORY -> ONE BUILD ARTIFACT             |
           | shared code, dependencies, configuration         |
           +----------+---------------------------+-----------+
                      |                           |
             start: HTTP server            start: job loop
                      |                           |
                      v                           v
            +--------------------+     +---------------------+
 requests ->|    WEB process     |     |   WORKER process    |
            | serves HTTP only   |     | runs queued jobs    |
            | own CPU/mem limit  |     | own CPU/mem limit   |
            | own threads + pool |     | own threads + pool  |
            +----+----------+----+     +----+-----------+----+
                 |          |               ^           |
                 |          | enqueue  pull |           |
                 |          v               |           |
                 |      +-----------------------+       |
                 |      |       JOB QUEUE       |       |
                 |      +-----------------------+       |
                 |                                      |
                 |   +------------------------------+   |
                 +-->|           DATABASE           |<--+
                     |  separate pool per process   |
                     +------------------------------+

One build artifact starts as two process types: web processes serve requests and enqueue jobs, while worker processes pull and run those jobs with their own CPU, memory, threads and database connection pool, so heavy background work cannot starve request handling.

How to implement it: entrypoints, boot paths and process declarations

Both process types run from the same build artifact, and only the start command differs. In a Procfile-style file, each line names a process type and its command: web starts the HTTP server, worker starts the queue consumer, and scheduler starts the periodic job runner. With containers, you build one image and give each deployment its own command or arguments. Using one image means every role runs identical code and dependencies, which rules out a whole class of "works in web, fails in worker" bugs.

Make the boot path explicit

Each role should take its own startup path through the application. A worker loads configuration, models and job definitions. It skips binding a port, loading the HTTP router and initializing web-only middleware such as session handling, CSRF protection or request logging. A web process never starts an in-process job runner or scheduler thread, even "just for small tasks." If it did, background work would run inside the same memory and CPU budget you are trying to protect.

Choose the role with a separate command or a required role flag or environment variable. If the role is missing, the process should exit at startup instead of falling back to a default. A silent default is how a web process ends up quietly consuming jobs.

Isolate the scheduler

Cron-style triggers belong in their own scheduler process type, run as a single replica. If every worker evaluated schedules, scaling workers from two to ten would fire each scheduled job ten times. If you need more than one scheduler replica for availability, have them take a distributed lock before firing, so only one instance runs each tick.

Copy# myapp/main.py: one entrypoint for every process type, same image.
# Procfile:
#   web:       python -m myapp.main web
#   worker:    python -m myapp.main worker
#   scheduler: python -m myapp.main scheduler   (run exactly one replica)
import sys

from myapp import config, models  # shared code every role needs

ROLES = ("web", "worker", "scheduler")


def run_web():
    # Only the web role imports the HTTP stack and web-only middleware.
    # It never starts a job runner or scheduler thread.
    from myapp.http import create_app, serve
    app = create_app(middleware=["sessions", "csrf", "request_logging"])
    serve(app, host="0.0.0.0", port=8000)


def run_worker():
    # No port binding, no router, no session or CSRF setup.
    from myapp.jobs import consume
    consume(queues=["default", "mail"])


def run_scheduler():
    # Fires periodic jobs onto the queue. Keep to one replica, or take a
    # distributed lock per tick if you run more than one.
    from myapp.schedule import run_forever
    run_forever()


def main():
    role = sys.argv[1] if len(sys.argv) > 1 else None
    if role not in ROLES:
        # No silent default: a missing role is a deploy bug, so fail fast.
        sys.exit(f"refusing to start: role must be one of {ROLES}, got {role!r}")
    config.load()
    models.init()
    {"web": run_web, "worker": run_worker, "scheduler": run_scheduler}[role]()


if __name__ == "__main__":
    main()

A single Python entrypoint that requires an explicit role and gives web, worker and scheduler separate boot paths. The comments show the matching Procfile declarations that start each role from the same build.

Sizing resources and connection pools per process type

Once each role boots on its own path, give it its own resource envelope. Web processes serve many short requests, so their limits should protect latency: steady CPU and modest, predictable memory. A worker's memory ceiling depends on its heaviest job multiplied by its concurrency. Setting limits per role means an out-of-memory worker gets killed and restarted while web replicas keep serving.

Size worker concurrency from memory, not core count. Subtract the process's baseline footprint from its memory limit, divide by peak memory per job, and round down to leave headroom. A worker with eight cores and room for three large exports should run three.

Every process holds its own database pool. The total across all replicas must stay below the database connection limit, minus a reserve for migrations and admin sessions.

Illustrative estimate. These inputs are assumed, not measured. The database allows 200 connections, with 20 reserved. Six web replicas each run two processes with pools of 5, which uses 60. Eight worker replicas with pools of 10 use 80. The total is 140, leaving 40 spare. Each additional worker replica costs another 10, so the twelfth replica uses the remaining budget and the thirteenth exceeds it.

Adding replicas does not remove connection exhaustion. It moves it. A connection pooler can cap the total, but past a certain worker count the database or a shared cache becomes the new contention point.

Scaling, queues and observability for each role

Web and worker processes need different scaling signals. A web process is saturated when requests queue in front of it, so request rate, p95 latency or CPU track its load closely. A worker can sit at low CPU while thousands of jobs wait, because it may be blocked on I/O. Workers should scale on queue depth and, better, on the age of the oldest job. Age reflects both arrival rate and throughput, and it maps directly to what users experience as "my export never arrived."

A single queue lets a burst of slow jobs, such as report generation, starve fast ones like password-reset emails. Split queues by priority or workload class and give each its own worker pool, so each pool scales on its own backlog.

Queue-based autoscaling oscillates. Workers drain the backlog, scale in, and the queue grows again. Use separate scale-out and scale-in thresholds and a cooldown longer than typical job duration. Scale-to-zero saves idle cost but adds boot time to the latency of the first job. Keep a minimum of one worker on latency-sensitive queues.

Tag every log line, metric and trace span with the process type and queue name. Track job duration, failure rate, retry count and enqueue-to-start latency. Workers have no HTTP endpoint to probe, so have them emit a heartbeat or update a last-progress timestamp. Alert when either goes stale.

Deploying workers safely: graceful shutdown and version skew

Workers have a different deployment lifecycle from web processes. When the orchestrator sends SIGTERM, a web process drains open requests in seconds. A worker may be partway through a ten-minute export. The handler should do three things. First, set a flag that stops the fetch loop from claiming new jobs. Second, let the current job finish if it can complete inside the termination grace period. Third, if it cannot, release or re-enqueue the job before the platform sends SIGKILL. For example: on SIGTERM, set shutting_down = true; the loop checks the flag before each fetch; a timer set a few seconds short of the grace period requeues whatever is still running.

Any job can still be killed mid-run, so make every job idempotent. Use upserts, deduplication keys, or a "processed" marker written in the same transaction as the side effect. If a task routinely runs longer than the termination window, split it into chunks that checkpoint progress and enqueue the next chunk.

During a rollout, old and new versions run at the same time. New web processes enqueue jobs that old workers pick up, and new workers drain jobs that old code enqueued. Payloads and schema changes must therefore stay compatible in both directions for at least one release. Suppose you rename user_id to account_id. Have producers write both fields, have consumers read account_id and fall back to user_id, and drop the old field one release later. Treat database migrations the same way: expand first, then contract.

Copyimport os, signal, threading

GRACE_PERIOD_S = 30    # must match the orchestrator's termination grace period
REQUEUE_MARGIN_S = 5   # requeue this many seconds before SIGKILL arrives
shutting_down = threading.Event()
current_job = None
# `queue` is a placeholder for your job queue client (claim/ack/release/push).

def on_sigterm(signum, frame):
    shutting_down.set()  # the fetch loop stops claiming new jobs
    t = threading.Timer(GRACE_PERIOD_S - REQUEUE_MARGIN_S, requeue_and_exit)
    t.daemon = True      # a clean finish exits without waiting for the timer
    t.start()

def requeue_and_exit():
    if current_job is not None:
        queue.release(current_job)  # another worker retries it; the job must be idempotent
    os._exit(1)  # exit now rather than race the requeued copy

signal.signal(signal.SIGTERM, on_sigterm)

def enqueue_export(account_id):
    # Release N: write both keys so old workers that still read user_id keep working
    queue.push("export", {"account_id": account_id, "user_id": account_id})

def account_id_from(payload):
    # Prefer the new key; fall back to the old key in jobs enqueued by old web code
    return payload.get("account_id") or payload.get("user_id")

def run():
    global current_job
    while not shutting_down.is_set():  # check the flag before every fetch
        job = queue.claim(timeout=5)
        if job is None:
            continue
        current_job = job
        export_for_account(account_id_from(job.payload))  # upsert-based, safe to rerun
        queue.ack(job)
        current_job = None
    # Loop exits once the in-flight job finishes; the process then ends normally

A worker that stops claiming jobs on SIGTERM, requeues the in-flight job shortly before the grace period ends, and produces and consumes a payload that tolerates the user_id to account_id rename across one release.

When to use it, when to avoid it, and the alternatives

Split the process types when background work measurably hurts request handling. Signs include p99 latency rising during batch runs, web processes restarting on out-of-memory errors caused by report generation, or CPU-heavy jobs starving request threads. Another reason is that jobs need a different resource profile from requests, such as more memory, longer timeouts, or a different concurrency model. In both cases, separate process types let each role get its own sizing and scaling rules.

Avoid the split when the app is small and the jobs are light, such as sending a few emails or refreshing a cache. A second role brings its own scaling policy, health checks, alerts, dashboards and capacity plan. Someone has to maintain all of that, and for low job volume the cost exceeds the benefit.

The split gives you runtime isolation, but the roles are still coupled. Web and worker run the same build, so a dependency upgrade, a memory leak in shared code or a boot-time bug ships to both at once. You cannot release one role on its own schedule, and a bad deploy can take down both.

Migration path and impact on cost and engineering time

A staged, reversible migration

Do the split in three steps. Each step changes one variable, and each can be undone by changing configuration rather than code.

1. Move job execution out of web. Stop running jobs in-process or in request threads. Start one worker process type that consumes every queue. Web only enqueues. Rollback means turning in-process execution back on.
2. Split queues. Route latency-sensitive jobs and bulk jobs to separate queues, and run a worker process per queue group. Rollback means pointing every worker back at all queues.
3. Add dedicated pools. Give each worker group its own concurrency settings, connection pool limits and scaling rules.

All roles ship from the same artifact, so the version is never what you roll back. You only reverse the process declarations and routing.

Cost and engineering time

The split costs two things:

• Always-on worker capacity. You pay for it even when queues are empty.
• Operational surface per process type. Each process type needs its own alerts, dashboards, shutdown tuning and deploy checks.

Illustrative estimate (assumed numbers, not a benchmark): monthly added cost = worker instances × instance price + worker groups × hours of upkeep per group × hourly engineering cost. With 2 instances at 50 per month, 2 groups and 3 hours each at 100 per hour, that comes to 100 + 600 = 700. Upkeep, not compute, dominates.

The split also returns engineering time:

• Fewer latency incidents. Batch jobs can no longer starve request threads.
• Faster debugging. Per-role metrics show which side is degraded.
• No premature service extraction. You avoid building a new runtime, pipeline and API contract before the job domain needs its own release cycle.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to monolith web and worker processes, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.