What is trunk-based development in a monolith, and why do long-lived branches cost more there?
Trunk-based development means every engineer integrates into a single shared branch, usually main, at least once a day. Changes land as small commits. Branches exist only to carry a pull request through review and are deleted within hours or a day or two. CI enforces one rule: main stays green. If a commit breaks the build or tests, the team reverts or fixes it right away, because everyone else is building on top of it.
In a system of small services, a long-lived branch mostly hurts the team that owns it. In a monolith it hurts everyone, for three reasons:
- One shared codebase. Every branch diverges from the same files that every other team is changing. Conflicts grow with the number of contributors, not just with your own change.
- A coupled build and test suite. A branch that compiles and passes on its own can still break when merged, because other code changed what it depends on. You only find out at merge time, when the context is gone.
- Hot files. Route tables, dependency-injection wiring, and schema or migration definitions are edited by nearly every feature. These files cause the bulk of merge conflicts and are the riskiest to resolve by hand.
Teams that share one deployable benefit most. The approach has a hard prerequisite: CI must be fast and reliable. If the pipeline is slow or flaky, engineers start batching changes, branches start living longer, and you lose the benefits of trunk-based development.
What types of feature toggles exist, and how long should each one live?
Once small commits land on trunk daily, unfinished work has to ship dark, and that means toggles. Not all toggles serve the same purpose, and treating them as one category is how a codebase ends up with hundreds of flags nobody dares delete.
| Type | Purpose | Owner | Expected lifespan | Removal trigger |
|---|---|---|---|---|
| Release | Hide incomplete or unproven code on trunk | Feature team | Days to a few weeks | Rollout reaches 100% and holds |
| Ops / kill switch | Shed load or disable a dependency during incidents | On-call / platform | Indefinite, reviewed periodically | Dependency or code path is retired |
| Experiment | Split traffic to compare variants | Product or data owner | Length of the experiment | Result is decided, losing branch deleted |
| Permission | Gate features by plan, role or tenant | Product / billing | Permanent | Entitlement model changes |
The table shows why mixing types causes debt. A release toggle should die within weeks. A kill switch should outlive most features. If both sit unlabelled in one flag store, nobody can tell which stale flag is safe to remove. Engineers then leave everything in place, and each surviving flag doubles the code paths someone has to reason about. Permission toggles also hide real business rules in a system built for temporary switches.
Treat every toggle as typed, owned code. Record its type and owner when you create it. Give release and experiment toggles an expiry date, and have CI or a scheduled job flag them once that date passes. Kill switches and permission toggles get review dates rather than expiry dates. Those dates force someone to confirm the flag is still wanted.
Where should toggle checks live in monolith code?
Evaluate a toggle once, as close to the edge as possible: the controller, the job handler, or the factory that wires up a service. At that seam, pick an implementation and inject it, so domain code receives a PricingCalculator and never asks which one it got. When if flag checks spread through domain logic, every branch doubles the paths your tests must cover. Removing the toggle later also means hunting through dozens of files instead of deleting one factory branch and one class.
Route all evaluation through a single internal interface rather than calling the flag vendor's client directly. Register each toggle there with its type, owner and expiry. That registry is what your CI or scheduled expiry check reads. It also gives you one place to swap flag stores without touching call sites.
Resolve every toggle a request needs at the start of the request and pass that snapshot along. If a flag flips mid-request and the code re-evaluates it, a user can write data through the new path and read it back through the old one. That inconsistency is hard to reproduce. Background jobs should also capture their flag snapshot when they are enqueued, not when they run.
Treat the flag store as a dependency that will fail. Give lookups a short timeout, cache the last known values in memory, and declare a default in code for each toggle. Choose the failure direction by toggle type:
- Release toggles fail closed, back to the old path.
- Kill switches fail toward the safe, degraded state.
- Permission toggles fail closed, so an outage never grants access.
type ToggleType = "release" | "kill-switch" | "permission";
interface ToggleDef { key: string; type: ToggleType; owner: string; expires: string; safeDefault: boolean }
// Single registry: CI expiry checks read this, and call sites never touch the vendor client.
// safeDefault encodes the failure direction chosen for each toggle type.
export const TOGGLES = {
newPricing: { key: "pricing.tiered-v2", type: "release", owner: "team-billing", expires: "2027-01-31", safeDefault: false }, // fail closed: old path
disableRecs: { key: "recs.kill", type: "kill-switch", owner: "team-discover", expires: "2027-06-30", safeDefault: true }, // fail toward degraded
bulkExportAdmin: { key: "admin.bulk-export", type: "permission", owner: "team-admin", expires: "2027-03-31", safeDefault: false }, // never grant on outage
} satisfies Record<string, ToggleDef>;
export type FlagSnapshot = Readonly<Record<keyof typeof TOGGLES, boolean>>;
// Adapter over the flag store; it enforces the timeout and serves last known values from memory.
interface FlagStore { get(key: string, timeoutMs: number): Promise<boolean | undefined> }
// Resolve every toggle once at the start of a request (or at enqueue time for jobs).
export async function snapshotFlags(store: FlagStore): Promise<FlagSnapshot> {
const entries = await Promise.all(Object.entries(TOGGLES).map(async ([name, def]) => {
const value = await store.get(def.key, 50).catch(() => undefined);
return [name, value ?? def.safeDefault] as const;
}));
return Object.freeze(Object.fromEntries(entries)) as FlagSnapshot;
}
interface PricingCalculator { quote(cartId: string): Promise<number> }
declare class LegacyPricing implements PricingCalculator { quote(cartId: string): Promise<number> }
declare class TieredPricing implements PricingCalculator { quote(cartId: string): Promise<number> }
// The seam: the only place this toggle is read. Removing it means deleting one branch and one class.
export function pricingFor(flags: FlagSnapshot): PricingCalculator {
return flags.newPricing ? new TieredPricing() : new LegacyPricing();
}
// Entry point: domain code receives a PricingCalculator and never asks which one it got.
export async function handleCheckout(req: { cartId: string }, store: FlagStore): Promise<number> {
const flags = await snapshotFlags(store);
return pricingFor(flags).quote(req.cartId);
}
A typed toggle registry carrying type, owner, expiry and a per-type safe default, resolved once per request into a frozen snapshot that a factory uses to inject the chosen PricingCalculator.
How does branch by abstraction work for large refactors?
Some changes are too large for a single toggle check. Replacing a persistence layer or a payment module touches hundreds of call sites. If you wrap each one in its own if on the same flag, the flag spreads across the codebase. Every reader then has to reason about both paths at every point, and removing it later becomes a refactor of its own. Branch by abstraction confines the decision to one place.
The sequence:
- Introduce an abstraction over the existing behaviour, such as a
PaymentGatewayinterface whose first implementation delegates to the current code. Behaviour does not change. - Move callers to the abstraction, one module per commit. Each step is small, reviewable and mergeable to trunk.
- Build the new implementation behind the same interface. It ships to production but receives no traffic yet.
- Switch with a release toggle at the single point where the implementation is chosen. You can roll out by percentage or tenant, and you can roll back by flipping one value.
- Delete the old implementation and the toggle, then decide whether the abstraction still earns its place.
The costs are real. You maintain two implementations for weeks. The extra layer makes stack traces deeper, and new contributors have to learn why it exists. During the migration, a "find usages" search shows a half-migrated codebase.
The biggest risk is stopping before step 5. An interface with one implementation and no prospect of a second is just indirection. Put the deletion date in the same ticket that introduces the abstraction.
// Step 1: the abstraction. Callers depend on this, not on a concrete module.
export interface PaymentGateway {
charge(orderId: string, amountCents: number): Promise<{ chargeId: string }>;
}
// Existing behaviour, wrapped unchanged. Delete in step 5.
class LegacyPaymentGateway implements PaymentGateway {
async charge(orderId: string, amountCents: number) {
return legacyBilling.chargeOrder(orderId, amountCents);
}
}
// Step 3: new implementation ships dark until the toggle routes traffic to it.
class LedgerPaymentGateway implements PaymentGateway {
async charge(orderId: string, amountCents: number) {
return ledgerClient.createCharge({ orderId, amountCents });
}
}
// Step 4: the ONLY place the release toggle is read.
// TOGGLE payments.ledger-gateway, expires 2026-12-15 (ticket PAY-412).
// Removing it means deleting this branch and LegacyPaymentGateway.
export function paymentGatewayFor(tenantId: string): PaymentGateway {
return flags.isEnabled("payments.ledger-gateway", { tenantId })
? new LedgerPaymentGateway()
: new LegacyPaymentGateway();
}
// Step 2: callers migrate one module at a time and never see the flag.
export async function checkout(order: { id: string; tenantId: string; totalCents: number }) {
const gateway = paymentGatewayFor(order.tenantId);
return gateway.charge(order.id, order.totalCents);
}
// Placeholders for existing code in the monolith.
declare const legacyBilling: { chargeOrder(id: string, cents: number): Promise<{ chargeId: string }> };
declare const ledgerClient: { createCharge(req: { orderId: string; amountCents: number }): Promise<{ chargeId: string }> };
declare const flags: { isEnabled(name: string, ctx: { tenantId: string }): boolean };
Branch by abstraction for a payment module: callers depend on a PaymentGateway interface, and a single factory function is the only code that reads the expiring release toggle that chooses between the legacy and new implementations.
How do you run dark launches and schema changes safely behind toggles?
A dark launch runs the new code path on production traffic while users still get the old path's result. In a monolith this is cheap to wire up: the toggle check calls both implementations in the same request. That is also what makes it risky.
For reads, use shadow reads: call the new path, discard its output, and log any difference from the old result. For writes, use dual writes: write through both paths and run a reconciliation job that compares the stores and reports drift. Divergence counts are your readiness signal, so watch them, not just error rates.
Isolation is the hard part. The shadow path must not send emails, capture payments, publish events or call partner APIs. Inject no-op or recording clients when the toggle is in shadow mode instead of relying on reviewers to spot every side effect. A missed one causes duplicate charges or corrupted rows that reconciliation reports only after the damage is done. Shadowing also adds database load and latency to every sampled request. Start with a small percentage and budget for the extra queries.
Schema changes follow expand/contract. First add new columns or tables that the old code ignores. Then ship code that works whether the toggle is on or off. Drop old structures only after the toggle is deleted. This keeps rollback a toggle flip rather than a migration.
Some changes cannot be toggled. Destructive migrations, long table rewrites, index builds and infrastructure changes have to be sequenced as separate, explicit deploy steps.
import logging
import random
log = logging.getLogger("shadow.pricing_v2")
class RecordingMailer:
"""Shadow-mode stand-in: records what would be sent, never sends."""
def __init__(self):
self.recorded = []
def send(self, to, subject, body):
self.recorded.append((to, subject))
def quote_total(order, flags, pricing_old, pricing_new, mailer):
# Users always get the old path's result.
result = pricing_old.total(order, mailer=mailer)
# Sample a small fraction of requests to cap extra DB load and latency.
# Toggle 'pricing_v2_shadow' is scheduled for removal once divergence stays at zero.
if flags.is_on("pricing_v2_shadow") and random.random() < flags.rate("pricing_v2_shadow"):
try:
# Inject a no-op client so the new path cannot email customers.
# Do the same for payment, event and partner-API clients.
shadow = pricing_new.total(order, mailer=RecordingMailer())
if shadow != result:
# Divergence count is the readiness signal, not just errors.
log.warning("divergence order=%s old=%s new=%s",
order.id, result, shadow)
except Exception:
# A failing shadow path must never break the real request.
log.exception("shadow error order=%s", order.id)
return result
A sampled shadow read that serves the old result, runs the new path with a recording mailer injected so it cannot send email, and logs any divergence without letting shadow failures affect the user.
How should you test and roll out code behind feature toggles?
Testing both states without testing every combination
For each active release toggle, run the affected tests with the toggle off and with it on. The main CI path should run with every toggle at its production default, which is usually off. That suite protects the code customers actually execute. Add one targeted job per toggle that turns it on and runs the tests touching that feature.
Do not try to test every combination. Ten boolean toggles produce 1,024 states, and nobody maintains that matrix. The cost of skipping it is that production can drift into a mix of toggles that no test environment has ever run. Short toggle lifetimes and few overlapping toggles in the same code path keep that risk small. Test the pairs you know interact, and treat any toggle whose lifetime keeps growing as a defect.
Code merged in the off state is easy to overlook. Reviewers skim it because it "doesn't run yet," and no real traffic exercises it. Plan for its first real use to be the moment its bugs show up.
Rollout mechanics
Set defaults per environment, for example on in development and staging and off in production, so the team uses the new path daily before customers do. In production, enable it for an internal cohort first, then by percentage.
Assignment has to be sticky. Hash a stable user or account ID so each user sees one consistent behavior across requests. Evaluate the toggle once per request and pass the result down the call stack. Re-checking it mid-request can mix old and new paths if the toggle flips while the request is running.
import hashlib
from dataclasses import dataclass
# Per-environment defaults: the team runs the new path daily before customers do.
ENV_DEFAULTS = {"development": True, "staging": True, "production": False}
# Production rollout for one release toggle. Delete this block when the toggle expires.
NEW_INVOICE_PDF = {
"name": "new_invoice_pdf",
"internal_accounts": {"acct_internal_001", "acct_internal_002"},
"percent": 10, # raise in steps once the internal cohort looks healthy
}
def bucket(toggle_name: str, account_id: str) -> int:
# A stable hash makes assignment sticky: the same account always gets the same bucket.
# Salting with the toggle name keeps different toggles from all hitting the same 10%.
digest = hashlib.sha256(f"{toggle_name}:{account_id}".encode()).hexdigest()
return int(digest[:8], 16) % 100
def is_enabled(toggle: dict, env: str, account_id: str) -> bool:
if env != "production":
return ENV_DEFAULTS[env]
if account_id in toggle["internal_accounts"]:
return True # internal cohort sees it first
return bucket(toggle["name"], account_id) < toggle["percent"]
@dataclass(frozen=True)
class RequestFlags:
new_invoice_pdf: bool
def handle_request(env: str, account_id: str) -> str:
# Evaluate once at the start of the request and pass the result down.
# Re-checking deeper in the stack could mix old and new paths if the toggle flips mid-request.
flags = RequestFlags(new_invoice_pdf=is_enabled(NEW_INVOICE_PDF, env, account_id))
return render_invoice(account_id, flags)
def render_invoice(account_id: str, flags: RequestFlags) -> str:
if flags.new_invoice_pdf:
return f"new PDF renderer for {account_id}"
return f"legacy PDF renderer for {account_id}"
A minimal release toggle with per-environment defaults, an internal cohort, sticky percentage rollout from a hashed account ID, and a single evaluation per request that is passed down the call stack.
When should you avoid trunk-based development, and what are the alternatives?
Trunk-based development assumes main is almost always releasable. That breaks down in three situations:
- Slow or flaky test suites. If the full suite takes an hour and fails intermittently, every merge either waits or lands unverified. One bad commit then blocks every team in the monolith.
- Weak review discipline. Small, frequent merges only work when reviews are fast and real. Rubber-stamped reviews put unfinished code on main without the safety a toggle is supposed to provide.
- Changes that cannot be toggled. A runtime upgrade, a build tool migration, or a dependency bump affects the whole process at once. No runtime flag can isolate it.
When these conditions apply, the alternatives trade off differently:
| Approach | Merge cost | Feedback speed | Operational overhead |
|---|---|---|---|
| Short-lived feature branches, rebased daily | Low if kept to a few days | Moderate; CI runs per branch | Low |
| Release branches | Rises with every cherry-pick | Slow for fixes | Medium; multiple lines to patch |
| Extract a module or service | High once, then low | Fast for the extracted part | High; separate deploys and contracts |
If you stay on trunk, treat toggle hygiene as a process rule. Every flag records an owner, a type, a creation date and an expected removal date. Removal is a tracked task, planned when the flag is created. CI or a lint rule warns when a flag passes its removal date.
Illustrative estimate, assumed inputs: 30 stale flags, each costing an engineer 20 minutes a month in confusion, extra test paths or debugging. That is about 10 hours a month of avoidable work. Removing a flag usually takes under an hour, so cleanup pays for itself within a few months.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to trunk based development monolith, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.