Async Request-Reply Pattern: HTTP 202 Accepted, Polling and Job Status

What the async request-reply pattern is and when to use it

The async request-reply pattern splits one slow HTTP call into a small protocol of short ones. The client submits work. The server validates it, records a job, and responds immediately with 202 Accepted and a Location header pointing to a job status endpoint. The client checks that endpoint until the job reaches a terminal state, then fetches the result from a URL the status response provides.

The pattern exists because the time a request may run is not set by your server alone. Every hop between client and handler enforces its own idle or total timeout: the client library, load balancers, API gateways, service meshes, CDNs. When any one of them gives up, the caller sees an error while your server may still finish the work, and nobody learns the outcome. Raising timeouts or holding connections open does not fix this. You rarely control every hop, especially for external consumers. Long-held connections tie up sockets and worker capacity. A deploy or dropped connection still loses the in-flight request, and the client cannot tell whether a retry is safe.

This guide is for backend and platform teams exposing long-running operations APIs to other services or external consumers.

Signals an endpoint should go async

• Latency varies widely with input size or depends on a slow downstream system.
• Requests intermittently fail with gateway timeouts even though the work completes.
• Clients retry timed-out calls and create duplicate side effects.
• The work queues behind limited resources such as GPUs, batch slots or rate-limited partners.

What is it? A protocol built from standard HTTP

Async request-reply is an API design pattern, not a library, framework or product. You implement it with ordinary HTTP semantics that every client and server stack already supports:

• 202 Accepted signals that the server has taken the request for processing but has not completed it.
• Location tells the client where to find the job.
• Retry-After tells the client how long to wait before checking again.
• A job status resource is a normal URL that represents the state of one unit of work.
• A result resource is a separate URL that holds the output once the work has finished.

Because the pattern is a contract rather than a dependency, it works the same in any language and behind any gateway or load balancer. The design decisions that matter are the shape of the job resource, the set of states it can be in, how the result is exposed, and how duplicate submissions are handled. The rest of this guide covers each of these.

Who should use it

The pattern fits the people who build and the people who consume operations that take longer than a single request can safely wait:

• API and backend engineers whose endpoints trigger work measured in tens of seconds, minutes or hours, such as report generation, data exports and imports, media transcoding, document processing, ML inference on queued hardware, or provisioning infrastructure.
• Platform teams that want one consistent way for many services to expose long-running work, so that gateways, SDKs and monitoring treat every slow endpoint the same way.
• Teams integrating with slow or rate-limited partners, where the latency of the downstream system is outside their control.
• Client and SDK authors who consume such APIs and need to write polling, backoff and timeout handling correctly.

It is a poor fit for operations that reliably finish within the shortest timeout on the request path, and for consumers whose tooling can only handle one request and one response. The section on when to avoid the pattern covers those cases and the alternatives.

How does it work? The moving parts

On the server side, an implementation has these components:

1. A submission endpoint that validates the request synchronously and rejects bad input with a 4xx.
2. A durable job store that records each job's ID, owner, state, timestamps, error and result pointer.
3. A queue or scheduler that hands work to workers.
4. Workers that claim jobs, run them, write the result, and move the job to a terminal state.
5. A status endpoint that reads the job record and returns its current state, cheaply and often.
6. Result storage with its own URL, access rules and lifetime.
7. Cleanup that expires old jobs and results.

The flow through these parts is: the client submits, the server persists a job and returns 202 with the job's location, a worker picks up the job and updates its state, the client polls the status endpoint (or receives a callback prompting it to check), and once the status is terminal the client follows the link to the result or reads the structured error. The next section walks through that exchange at the HTTP level.

How the exchange works end to end

The exchange splits one slow call into a fast write and a series of cheap reads.

Submission

The client sends POST /reports with its parameters. The server validates everything it can check cheaply and synchronously: authentication, schema, quotas, and whether referenced resources exist. A bad request should fail here with a 4xx, not hours later inside a worker. The server then persists a job record and enqueues the work. It writes the record first so that a status read can never return 404 for a job the client was just told exists.

The response is 202 Accepted with Location: /jobs/7f3a, an optional Retry-After: 5, and a body such as {"id": "7f3a", "status": "queued", "createdAt": "…"}. A 202 promises only that the request was accepted. It makes no promise about the outcome. That is why the body describes the job, not the result.

Processing

A worker claims the job, sets status to running, and may update a progress field. When it finishes, it writes the result or the error, then sets a terminal state, either succeeded or failed. It writes in that order so a client never sees succeeded with no result attached.

Status reads and retrieval

The client calls GET /jobs/7f3a. While the job is in progress, the server returns 200 OK with the current status and a fresh Retry-After. Once the job succeeds, the response includes a link such as "result": "/reports/91c2", and the client fetches the result from that link. The status endpoint stays small and cacheable, and the result gets its own URL, lifetime, and access rules.

CopyPOST /reports HTTP/1.1
Host: api.example.com
Content-Type: application/json

{"type": "sales", "from": "2025-01-01", "to": "2025-03-31"}

HTTP/1.1 202 Accepted
Location: /jobs/7f3a
Retry-After: 5
Content-Type: application/json

{"id": "7f3a", "status": "queued", "createdAt": "2025-04-02T10:15:00Z"}

GET /jobs/7f3a HTTP/1.1
Host: api.example.com

HTTP/1.1 200 OK
Retry-After: 10
Content-Type: application/json

{"id": "7f3a", "status": "running", "progress": 0.4}

GET /jobs/7f3a HTTP/1.1
Host: api.example.com

HTTP/1.1 200 OK
Content-Type: application/json

{"id": "7f3a", "status": "succeeded", "result": "/reports/91c2"}

GET /reports/91c2 HTTP/1.1
Host: api.example.com

HTTP/1.1 200 OK
Content-Type: application/json

The complete request-reply exchange: a 202 submission that returns a job record and Location header, status polls guided by Retry-After, and a terminal status that links to a separately addressable result.

Designing the job resource, result retrieval and idempotent submission

The job as a state machine

Model each job with explicit states: queued, running, and exactly one terminal state of succeeded, failed or cancelled. A terminal state never changes, so clients know when to stop polling and terminal responses can be cached. Record createdAt, startedAt and finishedAt. Add progress only if workers can report it honestly. A failed job should carry a structured error with a stable code, a readable message and a flag saying whether resubmitting could help. A bare string is not enough. A succeeded job carries the result link.

Returning the result

Idempotent submission

Accept an idempotency key or derive a natural key, such as report type plus date range. Store the key, the job ID and a hash of the payload. Keep the key longer than any client's retry window. If it expires while clients are still retrying, those retries create duplicate jobs. If the same key arrives with a different payload, reject the request instead of returning the old job. To handle concurrent duplicates, put a unique constraint on the key and return the existing job when an insert conflicts.

Job IDs should be random and unguessable, but that does not replace authorization. Check ownership on every status and result read, and return 404 to other callers.

Copy-- PostgreSQL 13+ (gen_random_uuid is built in). The app runs the submission statements in one transaction.
CREATE TABLE jobs (
  id          uuid PRIMARY KEY DEFAULT gen_random_uuid(),  -- random, unguessable ID
  owner_id    text NOT NULL,
  state       text NOT NULL DEFAULT 'queued'
              CHECK (state IN ('queued','running','succeeded','failed','cancelled')),
  created_at  timestamptz NOT NULL DEFAULT now(),
  started_at  timestamptz,
  finished_at timestamptz,
  error       jsonb,  -- {"code": ..., "message": ..., "retryable": ...}
  result_url  text
);

CREATE TABLE idempotency_keys (
  owner_id     text NOT NULL,
  idem_key     text NOT NULL,
  payload_hash bytea NOT NULL,  -- detects the same key sent with a different payload
  job_id       uuid NOT NULL REFERENCES jobs(id) DEFERRABLE INITIALLY DEFERRED,
  expires_at   timestamptz NOT NULL,  -- set well beyond the longest client retry window
  PRIMARY KEY (owner_id, idem_key)  -- keys are scoped per caller
);

-- Submission step 1: claim the key. $3 is a fresh UUID generated by the app.
-- A concurrent duplicate blocks on the unique index until the first transaction commits.
-- After that commit, this insert inserts nothing instead of creating a second job.
INSERT INTO idempotency_keys (owner_id, idem_key, payload_hash, job_id, expires_at)
VALUES ($1, $2, $4, $3, now() + interval '7 days')
ON CONFLICT (owner_id, idem_key) DO NOTHING
RETURNING job_id;

-- Step 2a: a row was returned, so this request owns the key. Create the job.
INSERT INTO jobs (id, owner_id) VALUES ($3, $1);

-- Step 2b: no row was returned. Look up the existing job.
-- If same_payload is false, the app rejects the request (for example with 422).
-- If it is true, the app returns the existing job.
SELECT job_id, payload_hash = $4 AS same_payload
FROM idempotency_keys WHERE owner_id = $1 AND idem_key = $2;

-- Every status or result read filters on the owner. Zero rows means the app returns 404.
SELECT id, state, created_at, started_at, finished_at, error, result_url
FROM jobs WHERE id = $1 AND owner_id = $2;

A PostgreSQL schema and set of queries for idempotent job submission, using a per-owner unique key, a payload hash and ON CONFLICT DO NOTHING to settle concurrent duplicates, with ownership checked on every status and result read.

Polling done well, and polling vs callback

Client side

Honor Retry-After when the server sends it. The server knows its queue depth, and the header lets it slow clients down without changing their code. When the header is absent, back off exponentially from about one second and add random jitter, so clients that submitted at the same moment do not poll in lockstep. Cap the interval and the total wait. A typical loop reads the status, stops on a terminal state, sleeps for Retry-After or the jittered backoff, and returns a timeout error to the caller once the deadline passes. A timeout should not be treated as a job failure.

Server side

Status reads outnumber submissions by a wide margin, so keep them cheap. Use a primary-key lookup on the job ID, and avoid joins and calls to the worker. An ETag on the status representation lets clients send If-None-Match and receive 304 Not Modified while nothing has changed. Rate-limit status reads separately from submissions so that aggressive pollers cannot use up submission capacity.

Polling vs callback

Keep the status endpoint as the source of truth, and add callbacks as an optimization. A callback should only tell the client to fetch the current status. That way, a lost, duplicated or reordered delivery cannot leave the client with the wrong state.

Copyimport random
import time

import requests

TERMINAL = {"succeeded", "failed", "cancelled"}


def wait_for_job(status_url, max_wait=300.0, base=1.0, cap=30.0):
    deadline = time.monotonic() + max_wait
    etag, attempt = None, 0
    with requests.Session() as s:
        while True:
            # Conditional GET: a 304 means nothing changed and costs the server little
            headers = {"If-None-Match": etag} if etag else {}
            resp = s.get(status_url, headers=headers, timeout=10)
            if resp.status_code == 200:
                etag = resp.headers.get("ETag")
                job = resp.json()
                if job["status"] in TERMINAL:
                    return job
            elif resp.status_code not in (304, 429, 503):
                resp.raise_for_status()  # real errors surface; throttling is retried

            # The server knows its load, so its Retry-After hint wins over local backoff
            retry_after = resp.headers.get("Retry-After", "")
            if retry_after.isdigit():
                delay = float(retry_after)
            else:
                # Exponential backoff with jitter so simultaneous clients spread out
                step = min(cap, base * 2 ** attempt)
                delay = random.uniform(step / 2, step)
                attempt += 1

            remaining = deadline - time.monotonic()
            if remaining <= 0:
                # Out of patience, not a job failure: the job may still finish
                raise TimeoutError(f"job still running after {max_wait}s: {status_url}")
            time.sleep(min(delay, remaining))

A Python polling loop that sends conditional requests with If-None-Match, follows Retry-After when the server provides it, otherwise backs off exponentially with jitter, and raises a timeout distinct from job failure once the total wait cap is reached.

Worker reliability, lifecycle and cancellation

Whatever queue or scheduler runs the work, assume each job can execute more than once. A worker can crash after doing the work but before acknowledging it, and the job will be redelivered. Make handlers idempotent by keying side effects on the job ID: upsert rather than insert, and pass the job ID as an idempotency key to downstream services that accept one.

Detecting stuck jobs

A job marked running whose worker has died stays running indefinitely unless something reclaims it. Give each claim a lease with an expiry, and have the worker renew it with heartbeats. When a lease expires, another worker can take over. Increment an attempt counter on every claim, and move the job to failed once it passes a limit. Otherwise a single poison input can loop through the worker pool forever.

Write ordering

Store the result durably first, then mark the job succeeded with a pointer to that result. If you reverse the order, a crash between the two writes leaves clients polling a successful job with nothing to fetch.

Expiry

Job records and results need TTLs, and the API reference should state them. For an expired ID, 410 Gone tells the client that the job existed and has been purged. 404 cannot distinguish an expired job from a mistyped ID. Returning 410 requires keeping a small tombstone after the job data is deleted.

Cancellation

Expose cancellation as DELETE on the job or as a cancel action, and document it as best-effort. A queued job can be cancelled cleanly. A running job can only be asked to stop at its next checkpoint, and it may already have produced side effects that the cancel does not undo. Report the final state truthfully: cancelled if the job stopped, succeeded if it finished first.

Orphaned jobs

Clients abandon jobs. If nobody has polled a job within a defined window, deprioritize it or cancel it, so that work nobody will collect does not consume capacity.

Copy-- PostgreSQL. $n are bind parameters supplied by the worker.

-- 1. Claim one job: a queued job, or a running job whose lease expired
--    (its worker died). SKIP LOCKED lets many workers poll without blocking.
UPDATE jobs
SET status = 'running',
    lease_owner = $1,                                 -- worker ID
    lease_expires_at = now() + interval '30 seconds',
    attempts = attempts + 1                           -- count every claim
WHERE id = (
  SELECT id FROM jobs
  WHERE (status = 'queued'
         OR (status = 'running' AND lease_expires_at < now()))
    AND attempts < 5                                  -- retry limit
  ORDER BY created_at
  LIMIT 1
  FOR UPDATE SKIP LOCKED
)
RETURNING id, payload;

-- 2. Heartbeat: renew the lease only if this worker still holds it.
--    Zero rows updated means the lease was lost; stop work.
UPDATE jobs
SET lease_expires_at = now() + interval '30 seconds'
WHERE id = $1 AND lease_owner = $2 AND status = 'running';

-- 3. Poison inputs: stop reclaiming jobs that exhausted their attempts.
UPDATE jobs
SET status = 'failed', error = 'max attempts exceeded'
WHERE status = 'running' AND lease_expires_at < now() AND attempts >= 5;

-- 4. Completion. Upsert keyed on job ID makes redelivery harmless.
--    The result is written first; the status flip comes only after.
INSERT INTO job_results (job_id, body) VALUES ($1, $3)
ON CONFLICT (job_id) DO UPDATE SET body = EXCLUDED.body;

UPDATE jobs
SET status = 'succeeded', result_ref = $1, lease_owner = NULL
WHERE id = $1 AND lease_owner = $2 AND status = 'running';

PostgreSQL statements for a lease-based job claim, heartbeat renewal, an attempt limit for poison inputs, and an idempotent completion that stores the result before marking the job succeeded.

When to avoid it and what the alternatives are

The pattern adds a job store, a status endpoint, cleanup and a second round trip for every result. That cost pays off only when work outlasts the timeouts between client and server. If an operation reliably finishes within the shortest timeout on the path, which may be a proxy or load balancer rather than your own server, a plain synchronous call is simpler and easier to debug. The same applies when consumers use tooling that assumes one request yields one response, such as shell scripts, spreadsheet connectors or some generated SDKs. Forcing a polling loop onto them moves complexity to the side least equipped to handle it.

A deadline with a fallback is often the best compromise. Most requests complete inline, and only the slow tail pays for the asynchronous machinery.

Impact on development cost and engineering time

The pattern moves complexity out of the request path and into infrastructure you now own. Even a minimal version requires:

• A job table with status, timestamps, the idempotency key, and a pointer to the result.
• A queue and workers, plus the deployment, scaling, and alerting that go with them.
• A status API with consistent fields, error shapes, and Retry-After behaviour.
• Cleanup jobs that expire old records and result blobs so storage doesn't grow without limit.
• Documentation and client helpers, because consumers have to learn a protocol rather than make a single call.

Each of these needs tests and on-call coverage. That ongoing ownership is the real cost, more than the initial build.

The pattern also removes some recurring work. Slow operations stop competing with gateway, load balancer, and client timeouts, so teams spend less time raising limits and investigating dropped connections. Idempotent submission makes retries safe, so a network blip doesn't create duplicate side effects that someone has to reconcile by hand. A single SDK helper that wraps submit-and-poll means each consumer writes the backoff, deadline, and error handling only once.

These savings depend on the job contract being uniform. If every endpoint defines its own status vocabulary and result location, the costs multiply and the client helpers stop being reusable. Build the job framework once, covering the schema, status endpoint, worker harness, cleanup, and SDK wrapper. New long-running endpoints then plug into it, and each one only has to supply its handler and its result type.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to async request reply pattern, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a route's URL or method automatically notifies the services that consume it — no hunting through repos for callers.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Consumers learn about a breaking change at review time rather than from production errors.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.