What is gRPC error handling and how do status codes work as a contract?

Every gRPC call, unary or streaming, ends with exactly one status: a code from a fixed set of 17 and an optional human-readable message. OK means success. Any other code means failure, and the response payload should not be trusted. This is the foundation of grpc error handling. The code is the machine-readable contract. The message is for humans and logs, and clients should never parse it.

The code set is small and stable across languages, so a Go server and a Java client agree on what NOT_FOUND means without sharing any error types. This lets you write client policy against codes instead of strings. Grouped by what the caller should do next:

Group Codes Client action
Caller-fixable INVALID_ARGUMENT, NOT_FOUND, ALREADY_EXISTS, FAILED_PRECONDITION, OUT_OF_RANGE Fix input or state; do not retry unchanged
Auth UNAUTHENTICATED, PERMISSION_DENIED Refresh credentials once, or surface to the user
Transient UNAVAILABLE, RESOURCE_EXHAUSTED, ABORTED, DEADLINE_EXCEEDED Retry with backoff, if the operation is safe to repeat
Server-side fault INTERNAL, UNKNOWN, DATA_LOSS, UNIMPLEMENTED Give up, alert, investigate

Clients receive some codes even if your handlers never return them. The gRPC library and transport produce these on their own:

  • DEADLINE_EXCEEDED when the deadline expires.
  • CANCELLED when either side cancels the call.
  • UNAVAILABLE when the connection fails or no backend is reachable.
  • UNIMPLEMENTED when the server does not recognise the method.

Many implementations also map an uncaught handler exception to UNKNOWN. Every client therefore needs a policy for these codes, whatever the server team documents.

Which gRPC status code should you return for commonly confused cases?

The canonical code definitions settle most disputes if you ask one question: what should the caller do next? Each pair below differs in the action it implies.

Confused pair Return the first when… Return the second when…
INVALID_ARGUMENT / FAILED_PRECONDITION The request is wrong regardless of system state (malformed email, negative page size) The request is valid but the system is not in the required state (deleting a non-empty bucket)
INVALID_ARGUMENT / OUT_OF_RANGE The value can never be valid The value could become valid as state changes (reading past the current end of a stream)
ABORTED / FAILED_PRECONDITION A concurrency conflict means the caller should retry the whole read-modify-write sequence (etag mismatch, transaction abort) Retrying is pointless until someone fixes the state
UNAVAILABLE / INTERNAL The failure is transient and the same call may succeed if retried An invariant broke and retrying will not help
UNAUTHENTICATED / PERMISSION_DENIED No valid credentials were presented The identity is known but not allowed

Return NOT_FOUND instead of PERMISSION_DENIED when admitting that a resource exists would itself leak information, such as tenant names, private repositories or user IDs. The cost is real. A legitimate user with a misconfigured role sees "not found" for something they know exists, and an operator cannot tell an access-control gap from a missing record without reading server logs. Make that trade deliberately for each resource type, not as a blanket default.

For unhandled exceptions, map to UNKNOWN or INTERNAL at a single interceptor. Because these codes collapse many different bugs into one bucket, the interceptor must log the stack trace and tag the metric with the exception class and handler. Without that tagging, your dashboards show one flat error rate and you lose the ability to see which class of bug is increasing.

How does the rich error model with google.rpc.Status work?

google.rpc.Status is a small protobuf message with three fields: an integer code, a developer-facing message string, and a repeated google.protobuf.Any details field. Each Any wraps a typed message with its type URL, so a client can unpack the details it recognizes and skip the rest. The type URL makes the payload self-describing, and unknown types do not break older clients.

On the wire, the status travels in HTTP/2 trailers as three separate pieces:

  1. Server handler returns an error built from a Status with details.
  2. gRPC server runtime writes grpc-status (the numeric code) and grpc-message (percent-encoded text). It also writes grpc-status-details-bin, which holds the whole serialized Status. Because of the -bin suffix, that value is base64-encoded in transit.
  3. Proxy, load balancer or gateway forwards the trailers, or fails to.
  4. Client runtime reads the trailers. It surfaces the code and message, and exposes the decoded details if the trailer arrived intact.

The duplication is deliberate. grpc-status and grpc-message are the baseline every gRPC implementation understands. The binary trailer is an optional extension, and the code and message inside it should match the plain trailers.

Step 3 is where things fail. Here is what can happen at each kind of intermediary:

  • Generic HTTP proxies may strip headers they don't recognize.
  • Transcoding gateways that convert to HTTP/JSON may drop binary trailers entirely.
  • Any hop that enforces a header-list size limit can reject or truncate a large details payload. Stack traces and long violation lists are common causes.

So treat details as an enhancement. Every client path must produce correct behavior from the code and message alone. Details should only refine that behavior.

+--------------------+      +--------------------------------------------+
| Server handler     |      | HTTP/2 trailers written by gRPC runtime    |
| returns Status{    |----->|  grpc-status: <numeric code>               |
|  code, message,    |      |  grpc-message: <percent-encoded text>      |
|  details: [Any..]} |      |  grpc-status-details-bin: base64(Status)   |
+--------------------+      +--------------------------------------------+
                                              |
                                              v
+-------------------------------------------------------------------------+
| Proxy / load balancer / gateway        (possible failure points)        |
|  [a] generic proxy strips headers/trailers it does not recognize        |
|  [b] HTTP/JSON transcoder drops grpc-status-details-bin                 |
|  [c] header-list size limit rejects or truncates large details          |
+-------------------------------------------------------------------------+
            |                                         |
            | all trailers intact                     | details-bin lost
            v                                         v
+-------------------------------+     +-----------------------------------+
| Client runtime                |     | Client runtime                    |
|  reads code + message         |     |  reads code + message only        |
|  + decodes details-bin:       |     |  -> must still behave correctly   |
|    unpack known Any types,    |     |  (details only refine behavior,   |
|    skip unknown type URLs     |     |   they are never required)        |
+-------------------------------+     +-----------------------------------+

A google.rpc.Status travels from the server handler as three HTTP/2 trailers, and because intermediaries can strip, drop or truncate grpc-status-details-bin, the client must behave correctly using only grpc-status and grpc-message.

Which standard error details should you use, and when?

Each detail type in google/rpc/error_details.proto answers a question the status code alone cannot. Pair each one with the codes whose meaning it refines.

Detail type Key fields Usual codes
BadRequest field violations (field, description) INVALID_ARGUMENT
ErrorInfo reason, domain, metadata Any
RetryInfo retry delay UNAVAILABLE, RESOURCE_EXHAUSTED, ABORTED
QuotaFailure violations (subject, description) RESOURCE_EXHAUSTED
PreconditionFailure violations (type, subject, description) FAILED_PRECONDITION
ResourceInfo resource type, name, owner NOT_FOUND, ALREADY_EXISTS, PERMISSION_DENIED
DebugInfo stack entries, detail INTERNAL, UNKNOWN
Help links Any, usually client-fixable errors
LocalizedMessage locale, message Any shown to end users

ErrorInfo is the most important type. Clients branch on reason within a domain, for example "API_KEY_EXPIRED" scoped to your service's domain. Once a client branches on a reason string, that string is part of your public API. If you rename it, the client's branch stops matching and falls through to generic handling, and nothing fails at compile time. Treat reasons the way you treat proto field names. Add new reasons freely, keep old ones stable, and deprecate them explicitly. Put variable data in metadata, not in the reason.

Detail size works in both directions. Over-detailed errors make trailers larger and can leak internal data. DebugInfo is the usual culprit, so strip it at your trust boundary. Sparse errors cause the opposite problem: clients start pattern-matching message text, and every wording change becomes a breaking change. A reasonable default is ErrorInfo on every error, plus one type-specific detail where the code calls for it.

import grpc
from google.rpc import code_pb2, error_details_pb2, status_pb2
from grpc_status import rpc_status

DOMAIN = "orders.example.com"

# Server side: one ErrorInfo for client logic, plus one detail type that matches the code.
def reject_quantity(context, qty):
    status = status_pb2.Status(code=code_pb2.INVALID_ARGUMENT, message="invalid order quantity")
    status.details.add().Pack(error_details_pb2.ErrorInfo(
        reason="QUANTITY_OUT_OF_RANGE",  # public API: add new reasons, never rename old ones
        domain=DOMAIN,
        metadata={"max_quantity": "100"},  # variable data belongs here, not in reason
    ))
    status.details.add().Pack(error_details_pb2.BadRequest(field_violations=[
        error_details_pb2.BadRequest.FieldViolation(
            field="items[0].quantity", description=f"must be 1-100, got {qty}"),
    ]))
    # No DebugInfo here: this error crosses the trust boundary to external clients.
    context.abort_with_status(rpc_status.to_status(status))

# Client side: branch on (domain, reason), never on the message text.
def handle_error(err: grpc.RpcError) -> str:
    status = rpc_status.from_call(err)  # None if the server sent no rich status
    if status is None:
        return "generic"
    for detail in status.details:
        if detail.Is(error_details_pb2.ErrorInfo.DESCRIPTOR):
            info = error_details_pb2.ErrorInfo()
            detail.Unpack(info)
            if info.domain == DOMAIN and info.reason == "QUANTITY_OUT_OF_RANGE":
                return f"cap quantity at {info.metadata['max_quantity']}"
    return "generic"  # an unknown or renamed reason silently lands here

A Python server packs ErrorInfo and BadRequest into a google.rpc.Status. The client uses grpc_status to unpack the status and branch on the (domain, reason) pair, so a renamed reason falls through silently to generic handling.

How do you implement rich errors on the server and a mapping layer on the client?

On the server, the pattern is the same in every language. Build a google.rpc.Status with the code and a developer-facing message. Pack each detail message into an Any, which records the detail's type URL. Then serialize the result into the grpc-status-details-bin trailer. In Go, status.New(codes.InvalidArgument, msg) followed by WithDetails(...) with an errdetails.BadRequest (one FieldViolation per bad field) and an errdetails.ErrorInfo (Reason, Domain, Metadata) does all of this in one call chain.

On the client, decoding has to be defensive:

  • Dispatch on type URL. Unpack only the detail types you know. Skip the rest without failing, because servers will add new detail types over time.
  • Treat missing details as normal. Proxies, older servers and errors raised by the gRPC library itself carry only a code and a message.

Put this decoding in one mapping function that runs on every RPC. It takes the status code plus ErrorInfo.reason and returns a typed domain error, such as QuotaExceeded or ValidationFailed with field violations attached. When no reason is present, it falls back to a generic error per status code. Application code then handles typed errors only and never reads message strings.

Language support for rich status is uneven. Some ecosystems ship helpers in the core library, others in a separate package, and some need manual trailer parsing. Polyglot teams should own a small shared pack/unpack utility for each language rather than let each service improvise one.

Centralizing this saves engineering time in three ways:

  • Faster debugging. Every error arrives with the same structure, so logs from different services can be compared directly.
  • Cheaper rewording. Changing an error message needs no client release.
  • One-place changes. Adding a new reason touches one mapping table instead of scattered string checks.
import grpc
from grpc_status import rpc_status  # pip: grpcio-status
from google.rpc import code_pb2, error_details_pb2 as ed, status_pb2

# Server: build google.rpc.Status, pack details into Any, abort (fills grpc-status-details-bin).
def reject_order(context):
    st = status_pb2.Status(code=code_pb2.INVALID_ARGUMENT, message="order rejected: bad quantity")
    st.details.add().Pack(ed.BadRequest(field_violations=[
        ed.BadRequest.FieldViolation(field="quantity", description="must be > 0")]))
    st.details.add().Pack(ed.ErrorInfo(reason="INVALID_QUANTITY", domain="orders.example.com"))
    context.abort_with_status(rpc_status.to_status(st))

# Client: typed domain errors. App code catches these and never reads message strings.
class DomainError(Exception): pass
class QuotaExceeded(DomainError): pass
class ValidationFailed(DomainError):
    def __init__(self, violations): super().__init__(violations); self.violations = violations

def map_rpc_error(err: grpc.RpcError) -> DomainError:
    st = rpc_status.from_call(err)  # None if no details trailer: proxies, old servers, library errors
    reason, violations = None, []
    for detail in (st.details if st else []):
        if detail.Is(ed.ErrorInfo.DESCRIPTOR):  # dispatch on type URL
            info = ed.ErrorInfo(); detail.Unpack(info); reason = info.reason
        elif detail.Is(ed.BadRequest.DESCRIPTOR):
            br = ed.BadRequest(); detail.Unpack(br)
            violations = [(v.field, v.description) for v in br.field_violations]
        # Any other detail type: skip silently; servers may add new ones later.
    code = err.code()
    if code == grpc.StatusCode.RESOURCE_EXHAUSTED and reason == "QUOTA_EXCEEDED":
        return QuotaExceeded(reason)
    if code == grpc.StatusCode.INVALID_ARGUMENT:
        return ValidationFailed(violations)  # empty list when no BadRequest was sent
    return DomainError(code.name)  # generic fallback per status code

# Usage: wrap every stub call once, e.g. in a client interceptor or helper.
# try: stub.PlaceOrder(req)
# except grpc.RpcError as e: raise map_rpc_error(e) from e

A Python server packs BadRequest and ErrorInfo into a google.rpc.Status, and a single client mapping function unpacks known detail types by type URL, skips unknown ones, tolerates a missing details trailer, and returns typed domain errors keyed on status code plus ErrorInfo.reason.

When should you retry, translate or pass through gRPC errors?

Retry is a property of the code and the operation. UNAVAILABLE usually means the request never reached a handler or was rejected before doing work, so it is the safest code to retry. Even so, retry only idempotent methods, use backoff with jitter, honor any RetryInfo delay, and stop once the remaining deadline cannot cover another attempt. INVALID_ARGUMENT, PERMISSION_DENIED and FAILED_PRECONDITION will fail the same way on every attempt until something outside the call changes, so never retry them.

DEADLINE_EXCEEDED needs care. A client-side deadline only means the client stopped waiting. The server may have committed the write a millisecond later. Retrying a non-idempotent CreatePayment after a timeout can charge twice unless the method accepts an idempotency key.

Code Retry? HTTP at gateway
UNAVAILABLE Yes, with backoff 503
RESOURCE_EXHAUSTED After RetryInfo delay 429
DEADLINE_EXCEEDED Idempotent methods only 504
ABORTED Retry the whole transaction 409
INVALID_ARGUMENT / FAILED_PRECONDITION Never 400
PERMISSION_DENIED Never 403

At service boundaries, translate downstream errors instead of passing them through. Consider a downstream inventory service that returns NOT_FOUND for a SKU. If the order service passes that code through unchanged, the caller reads it as "order not found." That code is misleading, and it also reveals the internal call graph. The order service should map the error to FAILED_PRECONDITION with an ErrorInfo reason such as SKU_UNAVAILABLE and log the original status. Avoid collapsing every downstream failure into INTERNAL, because that discards the information callers need to decide whether to retry.

import logging, random
import grpc
from grpc_status import rpc_status
from google.protobuf import any_pb2
from google.rpc import error_details_pb2, status_pb2

log = logging.getLogger("orders")
RETRYABLE = {grpc.StatusCode.UNAVAILABLE, grpc.StatusCode.RESOURCE_EXHAUSTED,
             grpc.StatusCode.DEADLINE_EXCEEDED}  # ABORTED: retry the whole txn, not the call

def next_delay(err, attempt, idempotent, time_left):
    """Seconds to wait before retrying, or None to stop."""
    if not idempotent or err.code() not in RETRYABLE:
        return None  # INVALID_ARGUMENT, PERMISSION_DENIED, FAILED_PRECONDITION never retry
    delay = random.uniform(0, min(2.0, 0.1 * 2 ** attempt))  # exponential backoff, full jitter
    status = rpc_status.from_call(err)
    for detail in (status.details if status else []):
        if detail.Is(error_details_pb2.RetryInfo.DESCRIPTOR):
            info = error_details_pb2.RetryInfo()
            detail.Unpack(info)  # server asked for a minimum wait; honor it
            delay = max(delay, info.retry_delay.seconds + info.retry_delay.nanos / 1e9)
    return delay if delay < time_left else None  # no time left for another attempt

def reserve_sku(context, inventory_stub, request):
    try:
        return inventory_stub.Reserve(request, timeout=context.time_remaining())
    except grpc.RpcError as err:
        log.warning("inventory.Reserve failed: %s %s", err.code(), err.details())
        if err.code() != grpc.StatusCode.NOT_FOUND:
            raise  # real code would map other codes too, not collapse them into INTERNAL
        # Downstream NOT_FOUND would read as "order not found"; translate it instead.
        detail = any_pb2.Any()
        detail.Pack(error_details_pb2.ErrorInfo(
            reason="SKU_UNAVAILABLE", domain="orders.example.com",
            metadata={"sku": request.sku}))
        context.abort_with_status(rpc_status.to_status(status_pb2.Status(
            code=grpc.StatusCode.FAILED_PRECONDITION.value[0],
            message="SKU is not available for ordering", details=[detail])))

A Python client helper that retries only idempotent calls on retry-safe codes, honoring RetryInfo and the remaining deadline, alongside a server handler that translates a downstream NOT_FOUND into FAILED_PRECONDITION with an ErrorInfo reason instead of passing it through.

When should you avoid rich error details, and what are the alternatives?

Rich details are not free, and some paths cannot carry them. Details travel in the grpc-status-details-bin trailer, so browser-facing routes, API gateways or proxies that strip or rewrite trailers can drop them silently, leaving the client with only a code and message. Small internal services where one team owns both ends gain little from the extra structure. A fleet where some languages lack mature helpers for packing and unpacking details will end up handling errors inconsistently.

Approach Survives trailer-stripping hops Machine-readable Main drawback
Status code + message only Mostly Code only Clients parse strings for detail
Result/oneof in response payload Yes Yes Every call returns OK, so code-based metrics and retries go blind
Ad-hoc metadata keys Depends on proxy Loosely No schema, so keys drift between services

Weigh the cost honestly. Rich errors add a dependency on the google.rpc protos in every client, helper utilities in each language, a review process for ErrorInfo reason strings so they stay stable, and translation code at every hop. That pays off when many teams consume an API and branch on reasons. It is overhead when two services talk only to each other.

Whichever model you choose, keep stack traces, hostnames and internal IDs out of messages and DebugInfo for external callers. Log the full error server-side with a correlation ID and return only that ID. Alert on error rate per status code, not on an aggregate count. A rise in UNAVAILABLE and a rise in INVALID_ARGUMENT need different responders.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to grpc error handling, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a route's URL or method automatically notifies the services that consume it — no hunting through repos for callers.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Consumers learn about a breaking change at review time rather than from production errors.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.