Async Request-Reply Pattern in Microservices: Status URLs First

When synchronous request-response stops working

The async request-reply pattern splits one long call into two short ones. The client submits work and gets an immediate acknowledgment with a reference to a status resource. Later it learns the outcome, either by checking that resource or by receiving a notification. No connection stays open while the work runs.

Teams usually adopt it after synchronous calls start failing in predictable ways:

• Infrastructure timeouts. Gateways, load balancers and proxies each enforce an idle or request timeout. An operation that takes longer than the shortest timeout in the path fails, even if the backend finishes successfully.
• Connections held for minutes. Each open request ties up a socket, a worker or thread, and often memory on every hop. Under load, slow requests exhaust connection pools and starve fast ones.
• Retries that duplicate expensive work. When a client times out, it cannot tell whether the server failed or is still working. A retry may start a second export or provision a second database.
• Coupled latency. The caller's response time becomes the callee's slowest dependency. So does the caller's availability. One degraded downstream service spreads latency back up the chain.

The pattern suits backend and platform engineers who own long-running operations, such as report and data exports, infrastructure provisioning, media processing and batch jobs. It is the right tool when the work regularly takes longer than infrastructure timeouts allow, when the caller can continue without the result, or when the work is costly enough that a duplicate run is a real problem. If an operation reliably finishes in a few hundred milliseconds, keep it synchronous.

How the flow works: 202 Accepted, Location, and the status resource

The flow has four steps, and only the first one is synchronous.

1. Submit. The client sends POST /reports with the job parameters. The API validates the input, writes an operation record (ID, pending state, timestamps) to a durable store, enqueues the work, and returns 202 Accepted with a Location: /operations/{id} header. Persist the record before you respond. If the API returns 202 and then crashes before the record exists, the client holds a URL that will return 404.
2. Execute. A worker picks up the job and moves the record through running to a terminal state: succeeded, failed, or cancelled. Workers write only to the status store. They never talk to the client directly.
3. Poll. The client issues GET requests against the status URL. While the operation is in progress, the response returns the current state, plus a Retry-After header if you want to set the polling pace.
4. Collect. A terminal response either embeds the result or links to it, for example /reports/{id}. A failed operation carries a structured error.

Copy Client ──POST /reports──▶ API ──write pending──▶ Status store
   ▲  ◀──202 + Location────┘                      ▲      │
   │                                              │      │
   │                          Worker ──update─────┘      │
   │                                                     │
   └──────────GET /operations/{id}───────────────────────┘
   ▲
   └─ optional callback: "operation {id} changed" ─▶ client GETs status URL

The status resource is the source of truth. Callbacks are an optional notification layered on top. A callback tells the client to look; it does not tell the client the answer. Callbacks can be lost, duplicated, or delivered out of order. Because the client confirms every callback against the status URL, none of those failures can corrupt its view of the operation.

Copy# 1. Submit: the API persists the operation record, then responds
POST /reports HTTP/1.1
Content-Type: application/json

{"type": "monthly-usage", "month": "2026-09"}

HTTP/1.1 202 Accepted
Location: /operations/op_7f3a9c
Content-Type: application/json

{"id": "op_7f3a9c", "status": "pending", "createdAt": "2026-10-25T09:00:00Z"}

# 2. Poll: still in progress, so the server sets the polling pace
GET /operations/op_7f3a9c HTTP/1.1

HTTP/1.1 200 OK
Retry-After: 5
Content-Type: application/json

{"id": "op_7f3a9c", "status": "running", "updatedAt": "2026-10-25T09:00:04Z"}

# 3. Collect: terminal state links to the result instead of embedding it
GET /operations/op_7f3a9c HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/json

{"id": "op_7f3a9c", "status": "succeeded",
 "updatedAt": "2026-10-25T09:01:12Z",
 "result": {"href": "/reports/rpt_51d2"}}

# Failure variant: same resource, structured error, no result link
# {"id": "op_7f3a9c", "status": "failed",
#  "error": {"code": "SOURCE_UNAVAILABLE", "message": "Usage data for 2026-09 not found"}}

An HTTP exchange showing the submit, poll, and collect steps: 202 Accepted with a Location header, an in-progress status response with Retry-After, and a terminal response that links to the result.

Designing the status resource and idempotent submission

The status resource carries the whole contract, so it needs a fixed shape. Every operation record should include:

• id: an opaque, stable identifier that never changes or gets reused. Clients store it, log it, and put it in support tickets.
• state: one value from an explicit state machine, with transitions enforced on the server.
• createdAt, updatedAt, completedAt: these let clients and operators tell a slow job from a stuck one.
• progress: include it only when the work has countable units, such as rows processed. A percentage based on guesswork misleads anyone watching it.
• error: a machine-readable code, a human-readable message, and a retryable flag. Free text alone forces clients to parse prose.
• result: a URL or reference to the output, not the output inlined. This keeps status responses small and cheap to poll.

Terminal states never change after they are reached. Clients can therefore stop polling and cache the result safely.

Submission must be idempotent, because a timed-out POST is indistinguishable from a lost one. Require a client-supplied idempotency key, scope it to the authenticated client so two tenants cannot collide, and retain it for a defined window, such as 24 hours.

The handler checks the (client, key) pair in a single atomic lookup-or-insert:

1. No record exists. Store the key with a hash of the request body, create the operation, and return 202.
2. A record exists and the body hash matches. Return the original 202 and the same status URL.
3. A record exists and the hash differs. Return 409 Conflict.

Silently reusing the earlier operation in the third case hides a client bug and returns a result for a request the client never made.

Copyimport hashlib, json, uuid

def submit_operation(conn, client_id, idem_key, body):
    # Hash a canonical form so key order and whitespace don't change the hash
    canonical = json.dumps(body, sort_keys=True, separators=(",", ":"))
    body_hash = hashlib.sha256(canonical.encode()).hexdigest()
    new_id = f"op_{uuid.uuid4().hex}"

    with conn:  # one transaction (psycopg2: commit on success, rollback on error)
        cur = conn.cursor()
        # Atomic lookup-or-insert on the (client, key) pair. A concurrent
        # duplicate blocks here until the first transaction commits.
        cur.execute("""
            INSERT INTO idempotency_keys
                (client_id, idem_key, body_hash, operation_id, expires_at)
            VALUES (%s, %s, %s, %s, now() + interval '24 hours')
            ON CONFLICT (client_id, idem_key) DO NOTHING
            RETURNING operation_id
        """, (client_id, idem_key, body_hash, new_id))

        if cur.fetchone():  # Case 1: first time this key is seen
            cur.execute("""
                INSERT INTO operations (id, client_id, state, created_at, updated_at)
                VALUES (%s, %s, 'pending', now(), now())
            """, (new_id, client_id))
            return 202, f"/operations/{new_id}"

        # Key already exists for this client (a cleanup job purges expired keys)
        cur.execute("""
            SELECT body_hash, operation_id FROM idempotency_keys
            WHERE client_id = %s AND idem_key = %s
        """, (client_id, idem_key))
        stored_hash, existing_id = cur.fetchone()

        if stored_hash == body_hash:  # Case 2: retry of the same request
            return 202, f"/operations/{existing_id}"
        # Case 3: same key, different body -> reject, never silently reuse
        return 409, None

A PostgreSQL-backed submission handler that resolves a client-scoped idempotency key with a single atomic INSERT ... ON CONFLICT and returns a new operation, the original status URL, or a 409 conflict when the key is reused with a different body.

Polling done properly

Polling is only expensive when the server gives clients nothing to go on. Include Retry-After on every in-progress status response. This lets the server tune poll frequency per operation from what it actually knows: a short value while a job is nearly finished, a longer one while it waits in a queue, and a larger one under load.

The client loop should work like this:

1. Wait for Retry-After if it is present. Otherwise use the current backoff interval, doubling it after each poll up to a cap.
2. Add random jitter to the wait so that clients submitted together do not poll in lockstep.
3. Send If-None-Match with the last ETag. A 304 means nothing changed and costs no body serialization.
4. Stop when the status is terminal or when an overall deadline passes. After the deadline, surface a timeout rather than polling forever.

On the server, a status read should be a primary-key lookup. It should never touch the worker or the result store. That makes it cheap to serve from a cache or replica, provided the ETag changes whenever the state does.

At completion, embed results only when they are small. For anything larger, return 303 See Other with a Location pointing to a separate result resource, or include a result link field. This keeps status responses uniformly small and lets results carry their own caching and expiry.

Watch the arithmetic. Ten thousand clients polling every second against a job that takes five minutes is three million requests for ten thousand units of work. That load can easily cost more than the work itself (illustrative estimate).

Copyconst sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

// Poll a status URL until a terminal state, a 303 to the result, or the deadline.
export async function pollStatus(statusUrl: string, deadlineMs = 10 * 60_000) {
  const deadline = Date.now() + deadlineMs;
  const maxBackoffMs = 30_000;
  let backoffMs = 1_000; // client fallback when the server sends no Retry-After
  let etag: string | undefined;

  while (Date.now() < deadline) {
    const res = await fetch(statusUrl, {
      headers: etag ? { "If-None-Match": etag } : {},
      redirect: "manual", // Node (undici) exposes the 303 instead of auto-following it
    });

    if (res.status === 303) {
      // Large result lives in its own resource with its own caching and expiry.
      return { resultUrl: new URL(res.headers.get("Location")!, statusUrl).href };
    }
    if (res.status === 200) {
      etag = res.headers.get("ETag") ?? etag;
      const body = await res.json();
      // Small results are embedded; larger ones may come back as a `result` link field.
      if (body.status === "succeeded" || body.status === "failed") return body;
    } else if (res.status !== 304) { // 304 = unchanged, no body to parse
      throw new Error(`status poll failed: HTTP ${res.status}`);
    }

    // Server hint wins. HTTP-date values parse to NaN here and fall back to backoff.
    const retryAfter = Number(res.headers.get("Retry-After"));
    const baseMs = retryAfter > 0 ? retryAfter * 1000 : backoffMs;
    backoffMs = Math.min(backoffMs * 2, maxBackoffMs);

    // Jitter breaks lockstep between clients; never sleep past the deadline.
    const waitMs = baseMs + Math.random() * baseMs * 0.5;
    await sleep(Math.min(waitMs, Math.max(0, deadline - Date.now())));
  }
  throw new Error(`operation not finished within ${deadlineMs} ms`);
}

A TypeScript polling loop that honors Retry-After, falls back to capped exponential backoff with jitter, sends If-None-Match to get cheap 304s, stops at an overall deadline, and handles both embedded results and a 303 redirect to a separate result resource.

Callbacks as an optimization: delivery, security, and reliability

Callbacks cut that load by letting the server announce completion. Treat them as an optimization layered on the status resource, not a replacement for it. The status URL remains the source of truth.

Delivery

Accept a callback URL per request in the submission body, or per subscription for clients that submit many jobs. Send a thin notification containing an event ID, the job ID, and the status URL, not the full result. Thin events keep sensitive payloads behind your normal authorization and make stale or reordered deliveries harmless. Receivers should return 2xx immediately and process asynchronously. Slow handlers cause sender timeouts and duplicate retries.

Security

Sign each request with HMAC over the timestamp concatenated with the raw body. Receivers recompute the signature, compare it in constant time, and reject timestamps outside a short replay window (the code example below shows both sides).

A callback URL is attacker-controlled input to an outbound request, so guard against SSRF. Resolve the hostname, block private, loopback, and link-local ranges, and connect to the IP you validated. Use an allowlist where practical. Before activating a subscription, confirm the receiver controls the endpoint by having it echo a challenge.

Reliability

Write events to a transactional outbox or durable queue in the same commit as the state change. Retry with exponential backoff and jitter. Delivery is at-least-once, so receivers deduplicate by event ID and re-read the status URL rather than trusting arrival order. After retries are exhausted, move the event to a dead-letter queue. Clients still poll occasionally to reconcile.

The cost is that every caller now runs a public, authenticated, highly available endpoint, which turns the caller into a server.

Copyimport hashlib
import hmac
import json
import time

SECRET = b"whsec_example_placeholder_not_real"  # per-subscription secret
REPLAY_WINDOW_SECONDS = 300  # reject anything older or newer than 5 minutes

def _mac(ts: str, body: bytes) -> str:
    # Sign timestamp + '.' + raw body so the timestamp cannot be swapped.
    return hmac.new(SECRET, ts.encode() + b"." + body, hashlib.sha256).hexdigest()

def sign(body: bytes) -> dict:
    # Sender: attach timestamp and signature headers to the outbound POST.
    ts = str(int(time.time()))
    return {"X-Webhook-Timestamp": ts, "X-Webhook-Signature": "v1=" + _mac(ts, body)}

def verify(headers: dict, raw_body: bytes) -> bool:
    # Receiver: verify the raw bytes before parsing JSON.
    ts = headers.get("X-Webhook-Timestamp", "")
    sig = headers.get("X-Webhook-Signature", "")
    if not ts.isdigit() or not sig.startswith("v1="):
        return False
    if abs(time.time() - int(ts)) > REPLAY_WINDOW_SECONDS:
        return False  # outside replay window
    # Constant-time comparison avoids leaking how many characters matched.
    return hmac.compare_digest(_mac(ts, raw_body), sig[3:])

# Thin event: IDs and the status URL only, never the result payload.
event = {
    "event_id": "evt_01HZX3K9Q2",
    "job_id": "job_7f3a",
    "status_url": "",
}
body = json.dumps(event, separators=(",", ":")).encode()
headers = sign(body)
assert verify(headers, body)
assert not verify(headers, body + b" ")  # any tampering fails
# After verifying: dedup on event_id, return 2xx, then GET status_url async.

Python HMAC-SHA256 signing over timestamp plus raw body for a thin callback event, and receiver verification with a replay window and constant-time comparison.

Operating long-running operations: expiry, cancellation, stuck jobs, and tracing

The status store is stateful infrastructure, and it needs the same care as any primary database. If it loses records, through a failover without replication or a cache used as the source of truth, clients end up holding operation IDs that nothing can answer for. They poll a URL that returns 404 for work that may have finished, so they resubmit or open a ticket. Give the store durable writes, backups, and a cleanup job.

Expiry

Keep operation records and results for a fixed, documented window, for example seven days after reaching a terminal state. After that, return 410 Gone if you keep a tombstone, or 404 if you don't. 410 tells the client the ID was valid and has expired, which is a different bug from a mistyped ID. Put the window in the status response itself, not only in the API docs.

Cancellation

Support DELETE on the operation or a dedicated cancel action, and define what happens to work already in progress. The usual approach is that cancel is a request, not a guarantee. The operation moves to cancelling, workers check a flag at safe checkpoints, and the final state is cancelled or succeeded if the work finished first. Clients must read the final state rather than assume the cancel worked.

Stuck jobs

A worker that crashes mid-job leaves a record in running indefinitely. Run a watchdog that moves operations past a per-type deadline to failed with a timeout reason. Emit counts per state and the age of the oldest running operation, and alert on that age.

Tracing

Assign a correlation ID at submission. Carry it through the queue message, the worker logs, and the callback headers, so one ID connects the whole request across services.

Alternatives, when to avoid it, and impact on cost and engineering time

Comparing the options

When to avoid it

• Fast operations. If an operation reliably finishes within client and gateway timeouts, a synchronous call is simpler.
• Purely internal traffic that already uses a broker. Reply queues give you correlation without adding HTTP status resources.
• No durable status store. If your team cannot run one, the status URL will lose state on restart. A job whose status is lost is worse than a timeout.

Cost versus time saved

The upfront work covers the status store, idempotency keys, a webhook delivery pipeline with retries, and a security review of the callback endpoints. In return, you get fewer timeout incidents and retries that do not duplicate work. You also get a single recovery and debugging path: anyone can fetch the status URL.

Start with polling. Add callbacks only when polling load or latency justifies them.

Illustrative estimate (assumed numbers, not a benchmark):

Polling load = active jobs ÷ poll interval

With an assumed 2,000 active jobs polled every 5 seconds, the load is 2,000 ÷ 5 = 400 status reads per second. Compare that figure with what your status store can serve before you build callbacks.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to async request reply pattern microservices, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.