What choreography and orchestration actually mean

In choreography, no service owns the process. Each service subscribes to events, does its own work, and publishes a new event. The order workflow exists only as the sum of those subscriptions. To answer "what happens after payment?", you read the consumers of PaymentCaptured across several repositories.

In orchestration, one coordinator holds the process definition. It calls each service as a step, records the result, and decides what runs next, including compensation when a step fails. The process can be read in one place, but the coordinator now depends on every service it calls.

CHOREOGRAPHY                          ORCHESTRATION

Order --OrderPlaced--> Payment                 +--> Payment
                         |                     |
                   PaymentCaptured        Coordinator --> Inventory
                         v                     |
                     Inventory                 +--> Shipping
                         |
                   StockReserved
                         v
                     Shipping

The two styles do not compete as whole-system architectures. Each one addresses a different coupling problem. Choreography keeps services unaware of each other, so teams can add consumers without coordinating releases. The cost is that the sequence of steps becomes coupled to event names and payloads that no single owner maintains. Orchestration makes the sequence explicit and puts it under one owner. The cost is that the coordinator is coupled to each service's API.

This article compares three concrete options:

  • Plain event-driven choreography over a message broker
  • A hand-built state machine orchestrator that persists its own state
  • A durable workflow engine such as Temporal, which persists workflow progress for you and resumes it after crashes
CHOREOGRAPHY                         ORCHESTRATION
(process = sum of subscriptions)     (process = one definition)

+----------+                         +------------------------------------+
|  Order   |                         |            Coordinator             |
+----------+                         |  1. Payment.capture                |
     |                               |  2. Inventory.reserve              |
     | OrderPlaced                   |  3. Shipping.create                |
     v                               |  on failure: compensate done steps |
+----------+                         +------------------------------------+
| Payment  |                              |            |            |
+----------+                         call |       call |       call |
     |                                    v            v            v
     | PaymentCaptured               +---------+ +-----------+ +----------+
     v                               | Payment | | Inventory | | Shipping |
+-----------+                        +---------+ +-----------+ +----------+
| Inventory |
+-----------+                        Services never call each other;
     |                               only the coordinator knows the order.
     | StockReserved
     v
+----------+
| Shipping |
+----------+

Each service reacts to the previous event and decides its own next step.

The same order flow in both styles: in choreography the sequence exists only in the chain of event subscriptions, while in orchestration a single coordinator holds the steps and compensation logic and calls each service directly.

How Temporal implements orchestration

A Temporal workflow is ordinary code (a function in Go, Java, TypeScript, Python, or another supported SDK) that the Temporal service runs durably. The service does not snapshot memory. It records an event history: workflow started, activity scheduled, activity completed with this result, timer fired, signal received. When a worker crashes or the workflow's state is evicted from its cache, another worker rebuilds the state by replaying the code against that history. Steps that already ran return their recorded results instead of executing again.

Replay only works if the code makes the same decisions every time it runs. That makes determinism a hard requirement. The common ways to break it:

  • Wall-clock time. Reading the system clock gives a different value on replay. Use the SDK's workflow time API.
  • Random values and UUIDs. Generate them through SDK-provided helpers or inside an activity, so the value is recorded.
  • Direct I/O. Network calls, database queries, and file reads belong in activities.
  • Non-deterministic iteration. For example, Go map ordering can change which branch runs first.

All side effects run in activities. Each activity has timeouts and a retry policy, so transient failures are retried by the engine rather than by hand-written loops.

Workers poll task queues for workflow and activity tasks, which lets you scale them independently and deploy them anywhere. Because the history is the state, you don't build a state machine or keep a status table.

History grows with every event. Long-running or chatty workflows should call continue-as-new, which starts a fresh execution that carries the current state forward and keeps the history bounded.

from datetime import timedelta
from temporalio import activity, workflow
from temporalio.common import RetryPolicy

BATCH_SIZE = 100  # keep each execution's history bounded

@activity.defn
async def charge_order(order_id: str, idempotency_key: str) -> str:
    # Direct I/O (payment API, database) goes here, never in workflow code.
    # The return value is recorded in the event history.
    return f"charge-{order_id}"

@workflow.defn
class BillingWorkflow:
    @workflow.run
    async def run(self, order_ids: list[str]) -> None:
        # Sort explicitly so iteration order is identical on every replay.
        pending = sorted(order_ids)
        batch, remaining = pending[:BATCH_SIZE], pending[BATCH_SIZE:]

        for order_id in batch:
            # workflow.uuid4() and workflow.now() are replay-safe;
            # uuid.uuid4() and datetime.now() are not.
            key = str(workflow.uuid4())
            workflow.logger.info("charging %s at %s", order_id, workflow.now())

            # On replay this returns the recorded result instead of re-running.
            await workflow.execute_activity(
                charge_order,
                args=[order_id, key],
                start_to_close_timeout=timedelta(seconds=30),
                retry_policy=RetryPolicy(
                    initial_interval=timedelta(seconds=1),
                    backoff_coefficient=2.0,
                    maximum_attempts=5,
                ),
            )

        if remaining:
            # Start a fresh execution with a new, empty history.
            workflow.continue_as_new(remaining)

A Temporal Python workflow that uses SDK helpers for time and UUIDs, runs side effects in a retried activity with a timeout, and calls continue-as-new after each batch to keep its event history bounded.

The same order saga in both styles

Take one order flow: reserve inventory, charge payment, schedule shipment, and undo earlier steps if a later one fails.

Choreography

The order service publishes OrderPlaced. Inventory consumes it and emits InventoryReserved. Payment consumes that and emits PaymentCharged or PaymentFailed. If payment fails, inventory must also subscribe to PaymentFailed and release the stock. Each compensation is a separate consumer of a failure event, so the rollback logic is spread across every service that took an action. No single component knows the order is "waiting on payment." To answer that, you build a projection that listens to all the events. Timeouts are hard to express here, because an event that never arrives produces nothing to react to. You need a scheduled sweeper that looks for stalled orders.

The pattern is only reliable with extra infrastructure. You need a transactional outbox so a database write and its event cannot diverge. You need dead-letter queues with replay tooling for poison messages. You need schema evolution rules so old consumers survive new event fields.

Temporal

One workflow function calls three activities in order. After each step succeeds, it pushes the matching undo action onto a list. On failure, it runs that list in reverse. Activity timeouts and retry policies are declared at the call site. The current step is the workflow's own execution state, which you can query directly.

Concern Choreography Temporal workflow
Compensation logic Spread across consumers One function, reverse order
Timeouts External sweeper Per-activity timeouts and retry policy
Current-step state Projection you build Workflow execution state
Delivery guarantees Outbox, DLQ, schema rules Event history plus activity retries
import { proxyActivities, defineQuery, setHandler } from '@temporalio/workflow';
import type * as activities from './activities';

// Timeouts and retries are declared once, at the call site, not in a sweeper.
const { reserveInventory, releaseInventory, chargePayment, refundPayment, scheduleShipment } =
  proxyActivities<typeof activities>({
    startToCloseTimeout: '30 seconds',
    retry: { initialInterval: '1 second', maximumAttempts: 5 },
  });

// The current step is workflow state that callers can query. No projection is needed.
export const currentStepQuery = defineQuery<string>('currentStep');

export async function orderSaga(orderId: string, amountCents: number): Promise<void> {
  let step = 'started';
  setHandler(currentStepQuery, () => step);

  // Each undo action is registered only after its step succeeds.
  const compensations: Array<() => Promise<void>> = [];
  try {
    step = 'reserving-inventory';
    await reserveInventory(orderId);
    compensations.push(() => releaseInventory(orderId));

    step = 'charging-payment';
    await chargePayment(orderId, amountCents);
    compensations.push(() => refundPayment(orderId, amountCents));

    step = 'scheduling-shipment';
    await scheduleShipment(orderId);
    step = 'completed';
  } catch (err) {
    // All rollback logic lives here. It runs in reverse order of the completed steps.
    step = 'compensating';
    for (const undo of compensations.reverse()) {
      await undo();
    }
    step = 'compensated';
    throw err;
  }
}

A Temporal TypeScript workflow for the order saga, with per-activity timeouts and retry policy, a compensation stack that unwinds in reverse on failure, and a query that exposes the current step from the workflow's own state.

Debugging, idempotency and change over time

Finding order 123

In choreography, no single component knows the order's state. To answer "where is order 123 stuck?" you need a correlation ID on every event, propagated by every consumer, and distributed tracing that spans the broker hops. A trace that just ends usually means a consumer dropped the message, failed to propagate context, or is backed up behind a DLQ. In Temporal, you look up the workflow execution by its ID (for example, order-123) and read its event history, which shows each activity scheduled, started, failed or retried, plus any pending timers or signals. The state is a record you read directly, not something you reconstruct from separate logs.

Idempotency is still your job

Neither model gives you exactly-once side effects. Brokers redeliver, so event consumers must deduplicate, typically on an event ID stored in the same transaction as the side effect. Temporal retries activities, and an activity can complete its external call but crash before reporting success, so it runs again. Pass an idempotency key derived from the workflow ID and step name to the payment provider or inventory API. The engine guarantees the workflow's own state transitions. It cannot guarantee anything about the systems your activities call.

Changing the flow

Adding a fraud check in choreography means deploying a new subscriber and possibly new event types that downstream services must learn to handle. In Temporal, workflow code is replayed against existing history. Changing step order on a running workflow therefore breaks executions already in flight. You gate the change with a version marker and keep the old code path until every execution started under it has finished.

// workflows.ts
import { proxyActivities, patched, workflowInfo } from '@temporalio/workflow';
import type * as activities from './activities';

const { reserveInventory, runFraudCheck, chargePayment } = proxyActivities<typeof activities>({
  startToCloseTimeout: '30 seconds',
  retry: { maximumAttempts: 5 }, // retries mean an activity can run more than once
});

export async function orderWorkflow(orderId: string, amountCents: number): Promise<void> {
  // Workflow ID such as "order-123" is stable across retries and replays.
  const { workflowId } = workflowInfo();

  await reserveInventory(orderId, `${workflowId}:reserve`);

  // Step added while executions were in flight. New runs record a marker and take
  // this branch. Replays of history without the marker get false and keep the old
  // command sequence. Once all old runs finish, switch to deprecatePatched().
  if (patched('add-fraud-check')) {
    await runFraudCheck(orderId);
  }

  // The key is derived from workflow ID + step, so a retried charge is deduplicated.
  await chargePayment(orderId, amountCents, `${workflowId}:charge`);
}

// activities.ts (excerpt)
export async function chargePayment(orderId: string, amountCents: number, idempotencyKey: string) {
  // The provider deduplicates on the key. Temporal cannot undo a charge that
  // succeeded before the worker crashed.
  const res = await fetch(`${process.env.PAYMENTS_URL}/charges`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json', 'Idempotency-Key': idempotencyKey },
    body: JSON.stringify({ orderId, amountCents }),
  });
  if (!res.ok) throw new Error(`charge failed: ${res.status}`);
}

A Temporal TypeScript order workflow that uses patched() to add a fraud check without breaking in-flight executions, and passes idempotency keys built from the workflow ID and step name so retried activities don't repeat side effects.

When to use each, who should own it, and when to avoid it

Versioning costs like that are part of the decision, but they matter less than a simpler question: who needs to know the whole process exists?

Choreography fits when a service publishes a fact, such as "order placed" or "price changed", and the consumers are independent teams that never report back. Analytics ingestion, notification fan-out and cache invalidation work this way. The publisher should not know or care who listens. Adding a consumer requires no change upstream, so each consuming team owns its own reaction.

Orchestration fits when the process has an owner who answers for its outcome. Signs include strict step ordering, waits for a human approval or a multi-day timer, and compensation that must undo several steps in reverse. One team owns the workflow definition and is accountable when an execution gets stuck.

Signal Lean toward Avoid it when
Broadcast fact, independent consumers Choreography Nobody can point to where the end-to-end flow is defined
Strict ordering or multi-step compensation Orchestration A single event fan-out already covers it
Human or timer waits Orchestration The workflow would reduce other teams' services to passive step handlers
Cyclic or emergent event chains Orchestration —

Treat a cycle as a warning. If service A reacts to B, which reacts to C, which triggers A again, you have an implicit workflow with no owner. In that case, move that logic into an orchestrator.

The opposite failure happens when an orchestrator calls into another team's internals step by step. Those teams lose control of their own contracts. In that case, the orchestrator should call their stable APIs, or the boundary should become an event.

The hybrid pattern: orchestrate inside a domain, choreograph across domains

Most production systems settle on a mix of the two styles. Inside one bounded context, such as fulfillment, a Temporal workflow orchestrates the process: reserve stock, charge the payment, book the carrier, and compensate if any step fails. When the workflow reaches a terminal state, it publishes a single domain event such as OrderFulfilled or OrderCancelled. Billing, analytics and notifications subscribe to that event and do their own work. The fulfillment workflow does not know these consumers exist.

This split puts each kind of coupling where it costs least. The owning team holds all the sequencing logic in one workflow, so when an order stalls they open one execution history and see every step, retry and failure. Other teams depend only on the event schema, which is a contract the owning team can version on purpose. Internal steps can be reordered or rewritten without telling anyone downstream.

Publishing the event is the last activity in the workflow, after the steps that commit state. It has to be an activity because workflow code must stay deterministic and cannot perform I/O directly. Activities run with at-least-once semantics, so it also needs idempotency. Suppose the broker accepts the message but the activity times out before it reports success. Temporal retries the activity and the event goes out twice. To handle this, derive the event ID deterministically from the workflow ID and outcome, not from a random value. Consumers then deduplicate on that ID.

Impact on cost and engineering time

Neither style is free. The costs land in different places and on different teams, which makes them easy to undercount.

Cost area Orchestration (Temporal) Choreography
Infrastructure Cluster plus persistence store, or a hosted service Broker, outbox relay, dead-letter queues
Ongoing ownership Worker deployments, task queue capacity, monitoring, usually owned by a platform team Schema compatibility tooling and dead-letter triage, spread across service teams
Learning curve Determinism rules and workflow versioning Idempotent consumers and reasoning about eventual consistency
Diagnosing a stalled process Workflow history shows the current step and pending activity Reconstructed from traces and logs across services

The engineering time Temporal saves comes from three places:

  • No hand-built state machines. You don't need a status column, a poller or a cleanup job for each workflow.
  • Fewer retry and timeout bugs. Retry and timeout policies are declared once, not reimplemented in every consumer, so there are fewer cases where a timer was forgotten.
  • Faster incident diagnosis. "Where is order 123 stuck?" becomes a lookup, not an investigation.

Illustrative estimate (assumed numbers, not a benchmark)

Net hours per year = (incidents × hours saved per incident) + (new workflows × hours of state-machine code avoided) − platform ownership hours

Assume:

  • 24 stalled-process incidents a year, with 3 hours saved on each
  • 4 new workflows a year, avoiding 40 hours of state-machine code each
  • 0.2 FTE to self-host the cluster, at 1,600 hours per FTE

That gives (24 × 3) + (4 × 40) − 320 = 72 + 160 − 320 = −88 hours.

With these inputs, self-hosting does not pay for itself. A hosted service, which cuts ownership hours, or a larger number of workflows changes the result. Plug in your own numbers before deciding.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to choreography vs orchestration microservices, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.