Microservice Timeout Budget: Propagating Deadlines Over HTTP

Why fixed per-hop HTTP timeouts cause cascading latency

Each service in a synchronous chain usually sets its own HTTP timeout and retry policy. That timeout is a guess about how long everything downstream should take. The guess breaks when someone adds a hop, moves a call, or adds a retry, and no caller is told.

The second problem is that the callee never learns when its caller gives up. Without propagated cancellation, a timed-out request keeps running downstream.

Illustrative estimate (assumed numbers, not a benchmark). Assume the chain is Edge → A → B → C. Every caller uses a 2s per-attempt timeout (t) and 2 retries (r), with retries run back to back and no cancellation passed downstream. There are H = 3 retrying hops.

• Edge gives up at: (r+1)·t = 6s.
• Last C call can finish at: H·r·t + t = 3·2·2 + 2 = 14s. That is 8s of work nobody will read.
• Worst-case calls reaching C per user request: (r+1)^H = 27.

During an incident, C is slow, so every layer times out and retries. That multiplies load on the service that is already struggling. This is how cascading latency grows into an outage.

A budget carries one deadline through the chain, and each hop spends from what remains. Stopping work once the deadline has passed is a correctness feature, not only a latency optimization. A response that arrives after the client gave up is wasted work, and it can still cause side effects nobody is waiting for.

This guide is for teams running synchronous, multi-hop HTTP request paths with user-facing latency SLOs.

How deadline propagation works across an HTTP call chain

The edge sets the budget once. It takes the user-facing latency target and subtracts its own work and a safety margin. The result is the deadline for the whole request. Every hop after that does the same calculation:

remaining = deadline − now − local overhead reserve

The reserve covers work the service still has to do after downstream calls return, such as serialization, writing the response, and logging. Every outbound HTTP timeout comes from remaining, never from a hardcoded constant. A fixed constant cannot know how much time upstream hops have already spent. If remaining is zero or negative, the service fails fast instead of making the call.

Plain HTTP has no universal standard header for this. Your team has to choose the header name, units, and semantics, and document them. Two encodings are common:

The relative form is usually safer across machines you don't control. It only stays correct if every hop remembers to subtract.

The diagram below is an illustrative estimate with assumed inputs (800 ms SLO, about 10 ms network allowance per hop):

CopyClient ── SLO 800ms
  │
Edge      budget 700 (100 reserved)
  │       spent 40, reserve 20, net 10 → send 630
Svc A     remaining 630
  │       spent 150, reserve 20, net 10 → send 450
Svc B     remaining 450
  │       spent 60, reserve 15, net 10 → send 365
Svc C     timeout = 365ms

Copy┌──────────┐
│  Client  │  user-facing SLO = 800 ms
└────┬─────┘
     │
     ▼
┌──────────┐  budget = 800 − 100 safety margin = 700 ms
│   Edge   │  700 − 40 spent − 20 reserve − 10 net = 630
└────┬─────┘
     │  X-Budget-Remaining-Ms: 630
     ▼
┌──────────┐  remaining = 630
│  Svc A   │  630 − 150 spent − 20 reserve − 10 net = 450
└────┬─────┘
     │  X-Budget-Remaining-Ms: 450
     ▼
┌──────────┐  remaining = 450
│  Svc B   │  450 − 60 spent − 15 reserve − 10 net = 365
└────┬─────┘
     │  X-Budget-Remaining-Ms: 365
     ▼
┌──────────┐  outbound timeout = 365 ms (derived, not hardcoded)
│  Svc C   │  if remaining ≤ 0 → fail fast, skip the call
└──────────┘

  Header name is a team-defined example; HTTP has no standard for it.
  Each hop: remaining = deadline − now − local overhead reserve

Illustrative estimate (assumed 800 ms SLO, 10 ms network allowance per hop) showing a relative remaining-milliseconds budget shrinking hop by hop as each service subtracts elapsed time, local reserve and network allowance before forwarding.

Implementing the budget inside a service

Handle the deadline once, at the start of the handler, before any business logic runs. Parse the incoming budget header. If it is absent or malformed, apply a per-endpoint default. If it exceeds a server-side maximum, clamp it to that maximum. Requests from external clients need stricter handling: ignore the header or cap it hard, because a public caller should never be able to make your fleet hold connections open for minutes. Compute the effective deadline as now + min(received, max) on the local monotonic clock. Because the header carries a relative duration, clock skew between hosts doesn't matter.

Bind that deadline to the request context or cancellation token, and pass the same context to every database query, cache call and outbound HTTP client. Some clients only accept a static timeout. For those, compute remaining − reserve at call time instead of using a configured constant. On each outbound request, re-encode the remaining budget into the header rather than forwarding the value you received, so the next hop sees time already spent.

Fail fast. Before each operation, compare the remaining time with a minimum useful threshold for that operation, roughly its typical fast-path latency. Below that threshold the call will almost certainly time out, so skip it. Return a distinct, documented error, such as a 504 with a machine-readable code like deadline_exceeded_upstream. Callers and dashboards can then tell budget exhaustion apart from a genuine downstream failure and avoid retrying it blindly.

Finally, verify that cancellation stops in-flight work rather than just discarding the result. Many drivers return early while the query keeps running on the database. Test this directly: send a request with a tiny budget, then confirm the query was cancelled server-side and the downstream service logged an aborted request.

Copy// Header carries a relative budget in ms, so clock skew between hosts is irrelevant.
const BUDGET_HEADER = "x-request-budget-ms";
const DEFAULT_MS = 2000;      // per-endpoint default when absent or malformed
const MAX_MS = 5000;          // server-side ceiling for internal callers
const EXTERNAL_CAP_MS = 1000; // hard cap for public clients

class DeadlineExceeded extends Error {
  status = 504;
  code = "deadline_exceeded_upstream"; // documented, distinct from real downstream errors
}

class Deadline {
  private readonly at: number;
  readonly signal: AbortSignal; // bind this to the request context
  constructor(budgetMs: number) {
    this.at = performance.now() + budgetMs; // local monotonic clock
    this.signal = AbortSignal.timeout(budgetMs);
  }
  remaining(): number { return Math.max(0, this.at - performance.now()); }
  // Fail fast: skip work that cannot finish in the time left.
  check(minUsefulMs: number): number {
    const left = this.remaining();
    if (left < minUsefulMs) throw new DeadlineExceeded();
    return left;
  }
}

function deadlineFromRequest(raw: string | undefined, isExternal: boolean): Deadline {
  const n = Number(raw);
  const ms = Number.isFinite(n) && n > 0 ? n : DEFAULT_MS;
  return new Deadline(Math.min(ms, isExternal ? EXTERNAL_CAP_MS : MAX_MS));
}

async function getInventory(dl: Deadline, sku: string) {
  const reserveMs = 20; // time we keep for our own response handling
  const left = dl.check(50) - reserveMs; // 50 ms ~ fast-path latency of this call
  // Re-encode the REMAINING budget, not the value we received.
  return fetch(` {
    signal: dl.signal, // aborts the in-flight request, not just the await
    headers: { [BUDGET_HEADER]: String(Math.floor(Math.max(0, left))) },
  });
}

A TypeScript sketch that parses, defaults and clamps an incoming relative budget header, binds it to an AbortSignal, fails fast with a distinct 504 error code, and forwards only the remaining budget on outbound calls.

Retries, hedging and fan-out under a shared deadline

A deadline turns retries, hedges and fan-out into spending decisions. Each pattern consumes the same remaining time differently, so each needs its own rule.

The retry check reads naturally in code: compute remaining = deadline - now, then retry only when remaining > expected_attempt + reserve. Without the reserve, the service retries into a deadline it cannot meet. The downstream does the work, and the caller has already given up on the result.

A deadline does not stop retry amplification. If three layers each retry twice, a single failure can become 27 calls at the bottom, and every one of them may fit its local budget. To prevent this, cap retries per request or enforce a retry budget, such as allowing retries to be at most a small fraction of total traffic. Ideally, only one layer retries.

For fan-out, reserve time before splitting the budget. You need enough time to merge results and to return a partial response when a branch misses the deadline. If the branches use the entire budget, the aggregator has no time left to degrade gracefully, and the request fails outright.

Copyimport time

RESERVE_S = 0.050      # time kept back for aggregation / fallback
MAX_ATTEMPTS = 3       # hard per-request cap, independent of the deadline


class RetryBudget:
    """Allow retries only up to a fraction of recent first attempts."""
    def __init__(self, ratio=0.1):
        self.ratio = ratio
        self.requests = 0
        self.retries = 0

    def can_retry(self):
        return self.retries < self.ratio * max(self.requests, 1)


def call_with_retries(send, deadline, expected_attempt_s, budget):
    # deadline is an absolute time.monotonic() value from the incoming request
    budget.requests += 1
    for attempt in range(MAX_ATTEMPTS):
        remaining = deadline - time.monotonic()
        if attempt > 0:
            # Retry only if another attempt plus the reserve still fits,
            # and the shared retry budget is not exhausted.
            if remaining <= expected_attempt_s + RESERVE_S or not budget.can_retry():
                break
            budget.retries += 1
        try:
            # Pass the remaining time downstream, minus our reserve.
            return send(timeout=remaining - RESERVE_S)
        except TimeoutError:
            continue
    raise TimeoutError("deadline or retry budget exhausted")

A budget-aware retry loop that retries only when the remaining time covers another expected attempt plus a reserve, and also respects a per-request attempt cap and a shared retry budget to limit amplification.

When to avoid deadline propagation and where the budget should stop

Deadline propagation assumes two things: that abandoning work is safe, and that every layer agrees on who owns the clock. Several common situations break one or both assumptions.

Writes with unknown outcomes

When the deadline expires during a non-idempotent write, the caller cannot tell whether the downstream committed. Retrying risks a duplicate charge, and giving up risks an unrecorded success. Attach an idempotency key so a retry is safe to repeat. If that is not possible, run a reconciliation process that resolves operations left in a pending state.

Where the budget ends

Teams need a written rule for which boundaries inherit the deadline.

Cancellation is often partial

Many HTTP clients, database drivers and databases do not stop server-side work when the caller gives up. A query can keep running after the client has returned an error. An expired budget frees the caller, but it does not necessarily free the downstream system.

Tuning and ownership

If you set budgets by guesswork, slow requests that would have succeeded become errors. Derive each hop's share from observed latency distributions, not from averages. Proxies, gateways and service meshes often enforce their own timeouts, and a shorter one silently overrides the propagated deadline. Designate one layer as authoritative and configure the others as looser backstops.

Small systems with shallow, stable call graphs may be served just as well by carefully chosen fixed timeouts.

Alternatives and complementary techniques

Each alternative bounds a different thing, so the choice depends on which failure you are trying to contain.

Depth-based timeouts break down when a service is reachable at several depths. It gets one value, so that value is too tight on one path and too loose on another. A propagated deadline adjusts per request.

Circuit breakers and concurrency limits work on a different axis. A deadline stops one request from waiting too long. It does nothing about thousands of requests piling onto a saturated dependency, each with a valid budget. Breakers and limits cap that aggregate load. Deadlines make them more effective because abandoned work releases its slots sooner.

Asynchronous designs remove the problem rather than manage it. If a step can be handled from a queue, take it off the request path. That is often simpler than tuning a budget across it.

Observability, rollout and impact on cost and engineering time

What to measure

Record three things per service: the budget received on the inbound request, the budget remaining at each outbound call, and deadline-exceeded counts broken down by hop and by whether the deadline expired locally or was inherited already spent. Add remaining budget as a span attribute on every client and server span. In a trace, this shows where the budget went. A timeout reported by service E can then be traced back to service B, which spent most of the budget before calling anything downstream. Without that attribute, the error appears at E and the investigation starts in the wrong place.

Rolling it out

Start at the edge and on two or three critical paths. Have services honor the deadline header when it is present and keep their existing timeouts when it is not. A missing header then changes nothing, so services can adopt it one at a time. Leave defaults loose until the metrics show how much budget each hop actually consumes, then tighten them.

Cost and payoff

Most of the cost comes up front. You need a shared middleware or library per language, header forwarding through every proxy and gateway (one that drops the header silently breaks propagation), conformance tests in CI, and an initial round of tuning.

The payoff is ongoing. Teams stop arguing over per-service timeout values. Incidents are attributed faster. Less work is wasted on requests the caller has already abandoned during overload. Timeouts stay correct when someone adds or reorders a hop in the call graph.

Copyimport time
from opentelemetry import metrics, trace

meter = metrics.get_meter("deadline")
budget_received = meter.create_histogram("deadline.budget_received", unit="ms")
budget_remaining = meter.create_histogram("deadline.budget_remaining", unit="ms")
exceeded = meter.create_counter("deadline.exceeded")

HEADER = "X-Request-Budget-Ms"  # placeholder name; must survive every proxy/gateway
FALLBACK_MS = 2000  # existing timeout, kept when the header is absent

def on_inbound(headers, service):
    """Run in server middleware inside the server span. Returns a monotonic deadline."""
    raw = headers.get(HEADER)
    if raw is None:
        return time.monotonic() + FALLBACK_MS / 1000  # missing header changes nothing
    budget = float(raw)
    budget_received.record(max(budget, 0), attributes={"service": service})
    trace.get_current_span().set_attribute("deadline.remaining_ms", budget)
    if budget <= 0:
        # Caller spent the budget before reaching us: not our latency.
        exceeded.add(1, attributes={"hop": service, "origin": "inherited"})
    return time.monotonic() + budget / 1000

def before_outbound(deadline, hop, out_headers):
    """Run in client middleware inside the client span. Returns timeout in seconds or None."""
    remaining_ms = (deadline - time.monotonic()) * 1000
    budget_remaining.record(max(remaining_ms, 0), attributes={"hop": hop})
    # Per-span attribute shows in the trace which hop consumed the budget.
    trace.get_current_span().set_attribute("deadline.remaining_ms", remaining_ms)
    if remaining_ms <= 0:
        # Budget ran out during our own work: count locally, skip the call.
        exceeded.add(1, attributes={"hop": hop, "origin": "local"})
        return None
    out_headers[HEADER] = str(int(remaining_ms))  # forward what is left
    return remaining_ms / 1000

Python middleware hooks using the OpenTelemetry API to record budget received, budget remaining per outbound hop, and deadline-exceeded counts split by local versus inherited, with a fallback timeout when the header is missing.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to microservice timeout budget, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.