What problem does JWT propagation between microservices actually solve?
When a request enters your system at the edge, a gateway authenticates the user and usually holds a JWT describing them. JWT propagation between microservices is the practice of carrying that identity onward as the request fans out to the services behind the gateway, so that each service can make its own authorization decision rather than trusting that someone upstream already did.
Every internal call has to answer two separate questions:
- Subject: which end user is this request on behalf of?
- Actor: which workload is actually making this call?
The actor question has a well-understood answer: mutual TLS or a platform workload identity proves that the caller is, say, the orders service and not something else on the network. Workload identity says nothing about the user, though.
The tempting shortcut is to forward the user's original JWT on every hop. That token is a bearer credential. Any service that receives it, or any log line or proxy that captures it, can replay it against any other service that accepts the same issuer and audience. A forwarded token also cannot say which service presented it. The receiver sees a valid signature for user 42 but cannot tell whether the request came from orders or from a compromised reporting pod reusing a token it saw earlier. So the token tells you who the user is, but not whether this caller has any right to act for them.
The practical goal is zero trust inside the cluster. Network location grants nothing. Each service verifies both the actor and the subject, and limits how far a leaked token can travel.
How do the three propagation patterns work, and when does each fit?
Each pattern below replaces the others. Mixing them on one call path makes it hard to know which tokens a service will receive and what it must validate.
1. Pass-through. Every service forwards the client's original access token unchanged. It adds no infrastructure and no extra latency. The cost is that a broad-audience credential reaches every service. Any service that leaks the token, or any logs that capture it, hands an attacker the user's full access.
2. Gateway minting. The gateway verifies the external token, then signs a short-lived internal token with only the claims services need. That makes the gateway a high-value signer. The model also assumes every internal request came through the gateway, so any path that bypasses it breaks the trust model.
3. Per-hop token exchange (RFC 8693). Before each call, the caller asks a token service to swap its token for one bound to the next audience, with narrower scopes. The act claim records who is acting on the subject's behalf:
Client ──► Gateway ──► Service A ──► Service B
sub=alice sub=alice sub=alice sub=alice
aud=api aud=svc-a aud=svc-b
act=gateway act=svc-a
└ act=gateway
Each hop costs a round trip, and the token service becomes a hard runtime dependency. Nested act claims also make tokens larger and policies harder to write, so many teams cap chain depth.
| Concern | Pass-through | Gateway minting | Token exchange |
|---|---|---|---|
| Blast radius | Full user access | Internal scope, all services | One audience, narrow scope |
| Latency | None added | One mint at the edge | One exchange per hop |
| Runtime dependencies | Identity provider keys only | Gateway signing key | Token service on every call |
| Revocation | Waits for the original token to expire | Short internal lifetime | Short lifetime, checked at each exchange |
| Gateway trust | Not required | Total | Partial |
| Operational cost | Low | Medium | High |
Token held or received at each hop (sub = subject, act = actor chain)
┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Client │──►│ Gateway │──►│ Service A │──►│ Service B │
├──────────────┤ ├──────────────┤ ├──────────────┤ ├──────────────┤
│ sub=alice │ │ sub=alice │ │ sub=alice │ │ sub=alice │
│ aud=api │ │ aud=api │ │ aud=svc-a │ │ aud=svc-b │
│ act=(none) │ │ act=(none) │ │ act=gateway │ │ act=svc-a │
│ │ │ │ │ │ │ └act=gateway│
└──────────────┘ └──────┬───────┘ └──────┬───────┘ └──────────────┘
│ token exchange │ token exchange
▼ ▼
┌─────────────────────────────────┐
│ Token service (RFC 8693) │
│ issues aud-bound, narrow token │
└─────────────────────────────────┘
Per-hop RFC 8693 token exchange: the subject stays alice end to end, each hop gets a token bound to the next audience, and the nested act claim records the chain of services acting on her behalf.
What must every receiving service validate in a JWT?
A gateway that validates tokens does not make the services behind it safe. Every service that acts on a JWT has to run the same checks itself. Otherwise, anything that can reach it on the internal network can hand it a forged or misdirected token.
A strict validator does the following:
- Signature. Read the
kidheader and use it to pick a key from a JWKS fetched from a trusted, preconfigured issuer URL. Never use a key or key URL that the token itself supplies. - Algorithm. Use a fixed allow-list, for example
RS256orES256only. Rejectnone. Bind each key to its algorithm so an attacker cannot switch the header toHS256and get the validator to treat the RSA public key as an HMAC secret. - Issuer. The
issclaim must exactly match the expected value. - Audience. The
audclaim must contain this service's own identifier. - Time. Enforce
expandnbfwith a small clock-skew allowance, typically under a minute.
The audience check is the one teams most often skip. Without it, a token minted for the orders service also passes validation at the payments service, because both trust the same issuer and keys. A compromised service, or anyone who captures a token from a log, can then replay it against a different service.
Validating in every service gives defense in depth. The cost is that the configuration must stay identical across languages and teams, and drifting audience or algorithm settings are a common source of real gaps. A shared validation library per language, or a sidecar that validates before traffic reaches the application, keeps the rules in one place and saves each team from rebuilding them.
# Illustrative config for an in-house shared JWT validation library.
# The schema is hypothetical; every service loads the same file shape
# and changes only service-specific values.
jwt_validation:
# Keys come only from this preconfigured issuer, never from the token.
# Token headers such as jku, x5u or jwk are ignored.
issuer: ""
jwks_uri: ""
jwks_cache_ttl_seconds: 300
# Select a key strictly by the token's kid. An unknown kid triggers at
# most one rate-limited JWKS refresh, then the token is rejected.
require_kid: true
# Fixed allow-list. 'none' and all HS* algorithms are absent.
allowed_algorithms: ["RS256", "ES256"]
# Each key must declare its own alg in the JWKS, and the token header
# must match that alg. This blocks RS256-to-HS256 confusion.
bind_algorithm_to_key: true
# This service's own identifier. Tokens minted for other services fail
# here even though they share the same issuer and signing keys.
audience: "payments-service"
# Missing claims are failures, not passes.
required_claims: ["iss", "aud", "exp", "nbf", "sub"]
clock_skew_seconds: 30
# Validation order applied by the library to every request:
# 1. Parse the header. Reject if alg is not in allowed_algorithms.
# 2. Look up kid in the cached JWKS. Reject if the key's alg differs
# from the header alg.
# 3. Verify the signature with that key.
# 4. Check that iss equals issuer exactly.
# 5. Check that aud equals audience, or contains it if aud is an array.
# 6. Check now < exp + skew and now >= nbf - skew.
# 7. On any failure, return 401 and log the reason, but not the token.
An illustrative configuration for a shared validation library that every service loads, with the service-specific audience and the step-by-step checks applied to each incoming JWT.
How do audience restriction and token exchange stop token replay and confused deputies?
Audience restriction limits where a stolen token works. If the gateway mints a token with aud: orders-api, the billing service rejects it because it compares aud against its own identifier. A token copied from a log line or a compromised orders pod therefore works against orders only. This also means orders cannot forward the user's token to billing. That leaves two options for the downstream call.
Exchange for a new token. Orders calls the authorization server's token endpoint using RFC 8693 parameters:
grant_type=urn:ietf:params:oauth:grant-type:token-exchangesubject_tokenset to the user's tokensubject_token_type=urn:ietf:params:oauth:token-type:access_tokenaudience=billing-apiscope=invoices:read- optionally
actor_token, carrying orders' own workload credential
Decoded, the result looks like sub: user-4821, aud: billing-api, scope: invoices:read, act: { sub: orders-api }.
Use the workload credential plus a signed user context. Orders authenticates as itself, for example with mTLS, and sends a separately signed assertion of the user. Billing verifies both.
The confused deputy case. A reporting service can read every tenant's invoices. User 4821 asks it for tenant B's report. If the reporting service queries with its own rights, the request succeeds. Billing must authorize against sub and check whether user 4821 can see tenant B. It uses act.sub only to confirm the caller is allowed to act on a user's behalf, and for audit. The resource owner enforces this check. The intermediary is not trusted to make it.
Caching exchanged tokens per subject, audience and scope saves round trips. Until a cached token expires, though, a revoked user stays active downstream, so keep the cache TTL short.
# Orders exchanges the user's token (aud: orders-api) for one scoped to billing.
# Orders authenticates to the authorization server with its mTLS workload cert.
curl -s -X POST "$TOKEN_ENDPOINT" \
--cert /var/run/secrets/workload/tls.crt \
--key /var/run/secrets/workload/tls.key \
--data-urlencode 'grant_type=urn:ietf:params:oauth:grant-type:token-exchange' \
--data-urlencode "subject_token=${USER_ACCESS_TOKEN}" \
--data-urlencode 'subject_token_type=urn:ietf:params:oauth:token-type:access_token' \
--data-urlencode 'audience=billing-api' \
--data-urlencode 'scope=invoices:read' \
--data-urlencode "actor_token=${ORDERS_WORKLOAD_JWT}" \
--data-urlencode 'actor_token_type=urn:ietf:params:oauth:token-type:jwt'
# The response's access_token, decoded (payload only, placeholder values):
# {
# "iss": "",
# "sub": "user-4821", <- billing authorizes against THIS user
# "aud": "billing-api", <- billing rejects any other audience
# "scope": "invoices:read",
# "act": { "sub": "orders-api" }, <- checked only for 'may act for users' + audit
# "exp": 1763300400
# }
#
# Billing-side checks, in order:
# 1. verify signature, iss, exp
# 2. aud == "billing-api"
# 3. act.sub is on the allowlist of services permitted to act for users
# 4. can sub (user-4821) read the requested tenant's invoices? <- confused-deputy guard
#
# If orders caches this token per (sub, audience, scope), keep the TTL short:
# a revoked user keeps working downstream until the cached token expires.
An RFC 8693 token exchange from the orders service, showing the decoded result with sub, aud and act claims and the checks billing runs to authorize against the user instead of the calling service.
Why shouldn't services trust headers like X-User-Id?
A header such as X-User-Id: 42 is a claim with no proof attached. The receiving service cannot tell whether the gateway set it or the caller did. Anything that can open a connection can send it: a compromised neighboring workload, a port-forward left open for debugging, a batch job, an internal route that skips the gateway. Stripping identity headers at the gateway protects only traffic that passes through the gateway, and attackers look for the paths that don't.
Zero trust replaces trust based on network location with proof on every hop. A signed internal token lets each service verify the user claim itself, using the checks from section 3. Mutual TLS proves which workload is making the call. The two are strongest together. If you bind the token to the mTLS connection, or check that the caller's workload identity is allowed to present that token, a stolen token is useless to an unauthorized workload.
That proof has an operating cost. Every workload needs a certificate from an identity system. Certificates must rotate before they expire, and the CA bundles that trust them must rotate too. Handshake failures are hard to debug, because the error rarely says whether the cause is clock skew, an expired intermediate certificate, or a wrong subject alternative name.
Misconfiguration can fail in either direction. A permissive mode or a missing validation middleware fails open: identity goes unchecked and nothing reports an error. An expired root certificate fails closed and breaks every call. Write explicit tests for both cases. A request that carries a forged header and no token must be rejected. Certificate rotation must run in staging before it first runs in production.
Operating propagated identity: keys, lifetimes, claims, logs, and async work
Keys and lifetimes
Cache the issuer's JWKS in every service. When a token arrives with an unknown kid, refresh, because that usually means a rotation just happened. Rate-limit that refresh per issuer. Otherwise a stream of forged kid values turns your fleet into an amplifier against the identity provider. Publish new keys before signing with them, and retire old keys only after the longest token lifetime has passed.
Keep internal tokens to minutes. A short lifetime is the cheapest form of revocation because it needs no extra calls, but a leaked token stays valid until it expires. A denylist closes that gap, but every service then has to receive shared state. Introspection gives the freshest answer, but it puts the issuer on every request path.
Claims
Carry only the claims downstream services actually decide on. Every claim rides every hop and lands in any proxy that logs headers, so names and email addresses don't belong in the token. Embedded roles and tenants save a lookup, but they stay stale until the token is reissued and they inflate headers. Per-request lookups are fresh, but they add latency and a dependency. A reasonable split is to embed stable facts like the tenant and look up volatile ones like permissions.
Logs and async work
Log identity, never the raw token: sub, act.sub, tenant, jti, trace_id.
A queue consumer often runs after the original token has expired, so don't put bearer tokens in messages. At enqueue time, either record the authorization decision or have the producer sign a minimal envelope such as {sub, tenant, action, resource_id, decided_at, exp} that the consumer verifies.
import time, uuid, logging
import jwt # PyJWT; ES256 also needs the 'cryptography' package
log = logging.getLogger("invoice-worker")
ENVELOPE_TTL = 24 * 3600 # must cover max queue delay plus retries, not the user's token lifetime
def sign_envelope(private_key_pem, kid, sub, tenant, action, resource_id):
# Producer calls this only AFTER its authorization check passed: the envelope records that decision.
now = int(time.time())
claims = {
"iss": "orders-api", "aud": "invoice-worker", # bind to one producer and one consumer
"sub": sub, "tenant": tenant, "action": action, "resource_id": resource_id,
"decided_at": now, "exp": now + ENVELOPE_TTL,
"jti": str(uuid.uuid4()), # lets the consumer dedupe and reject replays
}
# No bearer token, name or email goes into the message.
return jwt.encode(claims, private_key_pem, algorithm="ES256", headers={"kid": kid})
def verify_envelope(token, public_keys_by_kid):
# Pick the key by kid; an unknown kid means a rotation or a forgery, so fail closed.
kid = jwt.get_unverified_header(token).get("kid")
key = public_keys_by_kid.get(kid)
if key is None:
raise PermissionError("unknown envelope kid")
claims = jwt.decode(
token, key, algorithms=["ES256"], # pin the algorithm; never trust the header's alg
audience="invoice-worker", issuer="orders-api",
options={"require": ["exp", "sub", "tenant", "action", "resource_id", "jti"]},
)
if claims["action"] != "invoice.create": # the consumer only honors decisions it handles
raise PermissionError("unexpected action")
# Log identity fields only, never the envelope or any token.
log.info("envelope accepted", extra={
"sub": claims["sub"], "tenant": claims["tenant"],
"jti": claims["jti"], "decided_at": claims["decided_at"],
})
return claims
A producer signs a minimal authorization envelope with PyJWT at enqueue time, and the consumer verifies it by kid, pinned algorithm, audience, issuer and required claims, then logs only identity fields instead of the token.
Which pattern should your team start with, and when should you move to token exchange?
Start with the simplest pattern that holds for your topology. If one team runs a small cluster where every service shares the same owners and threat model, a gateway-minted internal token is usually enough. The gateway validates the external credential, issues a short-lived internal JWT, and every service validates it. You are trusting one boundary and one key set, and both match how the team actually works.
That changes when ownership fragments. In multi-team, multi-tenant, or regulated environments, start with audience-bound tokens and workload identity, and plan for exchange from day one. A token valid everywhere becomes a liability when one compromised service can replay it against another team's API.
Move to per-hop token exchange when any of these appear:
- Services live outside a single trust domain.
- Calls go to third parties or cross tenant boundaries.
- Auditors require a recorded delegation chain showing who acted for whom.
- Some services hold broad privileges, such as billing, admin, or data export.
| Environment | Starting pattern | Upgrade triggers | Main costs |
|---|---|---|---|
| Single team, trusted cluster | Gateway-minted token, validated at each hop | New teams, external callers | Key rotation, validation library |
| Multi-team or multi-tenant | Audience-bound tokens with workload identity | Cross-tenant calls, privileged services | Per-audience issuance, service registry |
| Regulated or cross-domain | Token exchange per hop | Already required | Token service availability, latency, operations |
The cost trade-off is front-loaded work versus deferred work. A token service and shared validation libraries take real engineering time up front. Skipping them means paying later in incident response and in authorization fixes patched one service at a time, and that later work tends to be less predictable.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to jwt propagation between microservices, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.