What is the saga pattern and why not use distributed transactions?
A saga replaces one distributed transaction with a sequence of local transactions. Each service commits its own step against its own datastore. If a later step fails, the saga runs compensating actions that semantically undo the earlier committed steps. A compensation is a new business operation, not a database rollback.
Two-phase commit is a poor fit across independently owned services. It requires every participant to hold locks while a coordinator decides. That couples availability, because one slow or crashed participant blocks the others. It also assumes all datastores and message brokers support a shared transaction protocol, which many do not. Teams that deploy on separate cycles also cannot reasonably agree on one coordinator's failure semantics.
A saga gives up three things:
- Atomicity across services. Partial states are visible until the saga finishes or compensates.
- Isolation between steps. Other requests can read or act on intermediate state.
- Immediate consistency. Correctness arrives eventually.
The running example is an order workflow:
| Step | Local transaction | Compensation | Notes |
|---|---|---|---|
| 1 | Create order (PENDING) |
Mark order CANCELLED |
Compensable |
| 2 | Reserve inventory | Release reservation | Compensable |
| 3 | Authorize payment | Void authorization, or refund if already captured | Compensable |
| 4 | Schedule shipment | Cancel shipment | Pivot: compensable only until the carrier accepts the parcel |
| 5 | Confirm order | None | Retriable; must eventually succeed |
The pivot is the point of no return. Before it, a failure triggers compensation. After it, the saga only moves forward, so every later step must be safe to retry until it succeeds.
How does a choreographed order saga work?
In a choreographed saga there is no coordinator. Each service listens for events, does its local transaction, and publishes the next fact.
For an order, the chain looks like this:
| Event | Published by | Consumed by | Reaction |
|---|---|---|---|
OrderCreated |
Order | Inventory | Reserve stock |
InventoryReserved |
Inventory | Payment | Authorize card |
PaymentAuthorized |
Payment | Shipping | Schedule shipment |
PaymentFailed |
Payment | Inventory, Order | Release stock; cancel order |
ShipmentScheduled |
Shipping | Order | Mark order confirmed |
The failure path works the same way. Inventory's handler for PaymentFailed looks up the reservation by order ID. If the reservation is still held, it releases it, then publishes InventoryReleased. If the reservation is already released, it acknowledges the event and does nothing. Order's handler independently moves the order from PENDING to CANCELLED. Payment does not know who reacts to its failure.
No single file or service describes the whole workflow. The workflow exists only as the union of subscriptions across repositories. To answer "what happens after payment fails?", you read every consumer of PaymentFailed.
The upside is real. Services deploy independently, the broker is the only shared dependency, and no central component can become a bottleneck or single point of failure.
The costs grow with the number of steps:
- Cyclic dependencies. Order consumes Shipping's events, and Shipping consumes events that trace back to Order. A schema change then has to ripple around the loop.
- Unowned failures. When Shipping rejects an address after payment, it is unclear whether Payment refunds, Order cancels, or both.
- Unnoticed stalls. If no one subscribes to a new failure event, the saga stalls silently.
// Inventory service: subscribes to OrderCreated and PaymentFailed
on OrderCreated(evt):
if reservations.exists(evt.orderId):
return ack() // duplicate delivery; already handled
if stock.reserve(evt.items):
reservations.save(evt.orderId, evt.items, status = HELD)
publish InventoryReserved { orderId: evt.orderId }
else:
publish InventoryUnavailable { orderId: evt.orderId }
on PaymentFailed(evt):
r = reservations.find(evt.orderId)
if r == null or r.status == RELEASED:
return ack() // idempotent: nothing left to undo
stock.release(r.items)
reservations.update(evt.orderId, status = RELEASED)
publish InventoryReleased { orderId: evt.orderId }
// Order service: subscribes to PaymentFailed and ShipmentScheduled
on PaymentFailed(evt):
order = orders.find(evt.orderId)
if order.status == PENDING:
orders.update(evt.orderId, status = CANCELLED)
publish OrderCancelled { orderId: evt.orderId }
on ShipmentScheduled(evt):
order = orders.find(evt.orderId)
if order.status == PENDING:
orders.update(evt.orderId, status = CONFIRMED)
// Payment service publishes PaymentFailed but holds no list of who reacts.
// The rollback path exists only because both handlers above subscribe to it.
Pseudocode handlers for the Inventory and Order services, showing how each one reacts on its own to PaymentFailed with an idempotent compensation, while Payment has no knowledge of the rollback.
How does orchestration with a durable workflow engine like Temporal work?
In orchestration, one coordinator owns the saga's state machine. It sends commands to each service, waits for replies, and decides what happens next. If a step fails, it runs the compensations in reverse order. The failure-routing problem from choreography goes away because one component is responsible for deciding.
Choreography (events) Orchestration (command/reply)
Order --OrderPlaced--> bus Orchestrator --ChargeCard--> Payment
Payment <-- bus Orchestrator <--Charged----- Payment
Payment --Charged----> bus Orchestrator --ReserveStock-> Inventory
Inventory <-- bus Orchestrator <--Reserved---- Inventory
A durable workflow engine such as Temporal records each workflow's decisions and activity results as an event history. After a crash, a worker rebuilds the workflow's position by replaying that history through the same code. You get crash survival without hand-written saga state tables.
A conceptual sketch:
- Charge payment. On success, push "refund payment" onto a compensation stack.
- Reserve inventory. On success, push "release reservation".
- Create the shipment.
- On any error, pop the stack and run each compensation, retrying until it succeeds.
This model has costs:
- Logic creep. The orchestrator tends to absorb domain rules and becomes a shared point of change across teams.
- Operational weight. The engine is another stateful system that you have to run, secure, upgrade, and back up.
- Determinism. Replay only works if workflow code avoids wall-clock reads, randomness, and direct I/O. Those must go through the engine's own APIs.
- Versioning. Instances that are still in flight replay against new code, so changes need explicit version branches.
- Extra hop. Each step routes through the engine, so expect some added latency on the critical path. This is an expectation from the design, not a measured figure.
// Conceptual saga orchestrator in the shape of a durable workflow.
// Illustrative pseudocode, not exact Temporal SDK calls.
// `acts` stands for engine-proxied activity stubs. Each call and its result
// is recorded in event history. On replay after a crash, recorded results
// are read back instead of re-executing the side effect.
interface OrderActivities {
chargeCard(orderId: string, cents: number): Promise<{ chargeId: string }>;
refundCharge(chargeId: string): Promise<void>;
reserveStock(orderId: string, skus: string[]): Promise<{ reservationId: string }>;
releaseReservation(reservationId: string): Promise<void>;
createShipment(orderId: string, address: string): Promise<void>;
}
async function placeOrderSaga(
order: { id: string; totalCents: number; skus: string[]; address: string },
acts: OrderActivities
): Promise<void> {
// Determinism: no Date.now(), Math.random(), or direct I/O in this function.
// All side effects go through activities so replay takes the same path.
const compensations: Array<() => Promise<void>> = [];
try {
const charge = await acts.chargeCard(order.id, order.totalCents);
compensations.push(() => acts.refundCharge(charge.chargeId));
const res = await acts.reserveStock(order.id, order.skus);
compensations.push(() => acts.releaseReservation(res.reservationId));
await acts.createShipment(order.id, order.address);
} catch (err) {
// Unwind in reverse order: the most recent successful step is undone first.
// Compensations must be idempotent. The engine's retry policy keeps
// retrying each one until it succeeds.
for (const undo of compensations.reverse()) {
await undo();
}
throw err; // Surface the original failure after compensating.
}
}
Illustrative pseudocode for an orchestrated order saga that pushes a compensation after each successful step and runs the compensations in reverse order when a later step fails.
How do you make saga steps and compensations safe to retry?
Compensations are business operations, not rollbacks. A refund is a new payment movement with its own fees, timing, and statement line; it does not erase the original charge. Releasing a stock reservation is only safe if nothing else has claimed that unit in the meantime. Design each compensation as a forward action with its own rules, not as an undo.
Under at-least-once delivery, both styles need three mechanics:
- An idempotency key per saga step. Derive it from the saga ID and step name, such as
order-123:charge. The payment service stores the key alongside the result. A retried request then returns the original outcome instead of charging twice. - A transactional outbox. The service writes its state change and an
outboxrow in the same local transaction. The row holds the event ID, aggregate ID, event type, payload, and a published timestamp. A separate relay publishes unsent rows. This removes the gap where the database commits but the broker publish fails, or the reverse. - Consumer-side deduplication. Each consumer records processed event IDs in the same transaction as its own side effects. Redelivered messages are then dropped.
Teams often pick choreography to "keep it simple" and then discover that every service needs all three mechanics, maintained independently. An engine retries activities for you, but those activities still need to be idempotent.
Some compensations fail or cannot be done at all. A parcel already handed to a carrier cannot be un-shipped, and a sent email cannot be unsent. When a compensation is impossible, or keeps failing after its retry budget runs out, move the order to a manual review queue with its full saga history. Otherwise it ends up silently inconsistent, with nobody assigned to fix it.
-- Outbox table: state change and event are committed together.
CREATE TABLE outbox (
event_id UUID PRIMARY KEY, -- consumers dedupe on this
aggregate_id TEXT NOT NULL, -- e.g. order ID
event_type TEXT NOT NULL,
payload JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
published_at TIMESTAMPTZ -- NULL until the relay sends it
);
-- Payment service stores one result per saga step.
CREATE TABLE payment_steps (
idempotency_key TEXT PRIMARY KEY, -- e.g. 'order-123:charge'
order_id TEXT NOT NULL,
status TEXT NOT NULL,
amount_cents BIGINT NOT NULL
);
BEGIN;
-- A retried command hits the primary key and inserts nothing,
-- so the step is recorded at most once.
INSERT INTO payment_steps (idempotency_key, order_id, status, amount_cents)
VALUES ('order-123:charge', 'order-123', 'CHARGED', 4999)
ON CONFLICT (idempotency_key) DO NOTHING;
-- Same transaction: no commit-without-publish gap.
-- If the step already existed, the WHERE NOT EXISTS skips a duplicate event.
INSERT INTO outbox (event_id, aggregate_id, event_type, payload)
SELECT '7d4e1c2a-5b3f-4a9e-8c61-2f0b9d3e4a10', 'order-123', 'PaymentCharged',
'{"orderId":"order-123","amountCents":4999}'::jsonb
WHERE NOT EXISTS (
SELECT 1 FROM outbox
WHERE aggregate_id = 'order-123' AND event_type = 'PaymentCharged'
);
COMMIT;
-- Relay polls unsent rows, publishes them, then sets published_at.
SELECT event_id, event_type, payload FROM outbox
WHERE published_at IS NULL ORDER BY created_at LIMIT 100;
A PostgreSQL outbox table and a single transaction that records a payment step under its idempotency key and queues the matching event, so a retry neither charges twice nor publishes twice.
How do you handle isolation anomalies, timeouts, and stuck sagas?
Sagas give up isolation, so other requests can see an order that is only half finished. Four countermeasures cover most order workflows:
- Semantic lock. The order sits in a
PENDINGstate, and other operations such as ship, cancel, or edit refuse to touch it until the saga commits or compensates. - Commutative updates. "Reserve 2 units" can be applied in any order alongside other reservations. "Set stock to 8" cannot, and it loses concurrent writes.
- Reread before acting. Check current stock or price at the step that uses it, not the value captured when the saga started.
- Risky step last. Put the least-compensable step, such as capturing payment or handing the order to a carrier, at the end. Nothing after it then needs undoing.
Stuck sagas are where the two styles differ most.
| Concern | Choreography | Orchestration |
|---|---|---|
| Detecting a step that never answered | Each service runs its own timer or a sweeper job that scans for orders stuck in PENDING past a deadline. Timeout logic is spread across services. |
The workflow starts a durable timer next to the step call. When the timer fires, the workflow compensates. |
| Investigating one stuck order | The operator stitches together correlation-ID traces across several topics and service logs. | The operator reads one workflow history: each step, its input, its retries, and its timers. |
| Adding a fraud check before payment | Payment stops reacting to InventoryReserved and listens for FraudCleared instead. A new rejection event needs inventory subscribers that release stock. Several teams have to coordinate their deploys. |
Add one step and its compensation branch in the workflow. In-flight workflows must be versioned. |
The fraud-check row shows the cost difference. In choreography, a change to the flow means changing several services and the contracts between them. In orchestration, the same change usually stays inside one codebase.
When should you use orchestration or choreography, and when should you avoid each?
Making that change in one workflow definition, deployed by one team, is the main argument for orchestration. Orchestration is the better default once an order saga has more than three or four steps, or any compensation a customer or finance team would notice, such as a refund, a voided shipment label, or a released reservation. Throughput and architectural fashion do not decide it. What decides it is where the saga's state and failure handling should live. Choreography spreads that state across services and the broker. Orchestration keeps it in one place that you can query.
The choice should follow ownership. If one team owns the end-to-end order process, it should own an orchestrator. If no single team owns the outcome, and services only react to facts like "order placed," choreography avoids creating an owner that doesn't exist.
Avoid choreography when support staff need to answer "where is order 4812 stuck?" Reconstructing that from event logs across services is slow and error-prone. Avoid orchestration for one-hop reactions such as sending a confirmation email. A workflow there adds a dependency and an operational surface for no gain.
| Factor | Favors choreography | Favors orchestration |
|---|---|---|
| Step count | 2–3 steps | 4+ steps or branching |
| Rate of flow changes | Rare | Frequent, product-driven |
| Process ownership | Shared or none | One team |
| Per-order visibility | Not needed | Required for support or audit |
| Operational maturity | Strong with the broker, DLQs and replay | Can run and upgrade a workflow engine |
What are the alternatives and hybrid designs?
The most common hybrid keeps the core order saga orchestrated (reserve inventory, authorize payment, create shipment) and publishes events for side effects such as confirmation emails, analytics, and loyalty points. This works because those consumers cannot veto the order and never need compensation. If an email fails, the order is still valid.
The boundary rule is simple. Any step that can change the order's outcome or requires compensation belongs in the workflow. Everything else consumes facts the workflow emits after a state change is durable. If you let a "side effect" consumer cancel orders or call back into the saga, you now trace failures through both workflow history and broker logs, and you pay the debugging costs of both styles.
Other options are worth considering before you adopt either style:
| Option | Fits when | Engineering time | Operational cost |
|---|---|---|---|
| Two-phase commit | Every participant shares a transactional resource, such as one database | Low; the resource manager does the work | Locks held during coordinator failure; availability is coupled across participants |
| Merge steps into one service | Steps change together and are owned by one team | Low; a local transaction replaces compensations | A larger service and deploy unit; scaling is coupled |
| Hand-rolled orchestrator with a state table | Few steps, and you want no new infrastructure | High; you build timers, retries, recovery, and versioning yourself | Ongoing maintenance of that machinery and its stuck-row tooling |
Merging is often the cheapest fix. If two steps always need compensating together, the service boundary between them may be wrong.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to saga pattern orchestration vs choreography, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- The dependency graph answers "what does this touch?" in seconds instead of a manual audit.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- You can size a change's blast radius before merging, avoiding surprise cross-team firefighting.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.