What is gRPC REST transcoding and how does it work?
gRPC REST transcoding is the translation of an HTTP/JSON request into a gRPC call, and of the protobuf response back into JSON, driven by declarations in your .proto files. The proto stays the single source of truth. The transcoder reads the google.api.http annotations on each RPC to learn which HTTP verb and URL path map to which method.
The request lifecycle
- Match. An incoming request such as
GET /v1/users/42?view=FULLis compared against the path templates declared in the annotations. The verb and path together select one gRPC method. - Merge. The transcoder builds the request message from three sources:
- Path variables like
{user_id}populate the named fields. - The JSON body fills either the whole message or a single designated field.
- Query parameters fill any remaining fields not bound by the path or body.
- Path variables like
- Call. The assembled message is sent to the backend over a normal gRPC connection. The backend does not know the caller spoke REST.
- Serialize. The response message is rendered using the canonical proto3 JSON mapping. Field names become lowerCamelCase, enums are emitted as names, 64-bit integers are emitted as strings, and well-known types such as
Timestampbecome RFC 3339 strings.
Where it runs
- grpc-gateway: A protoc plugin that generates a Go reverse proxy. It runs as its own process or inside your Go server, between REST clients and the gRPC backend.
- Envoy's gRPC-JSON transcoder: A filter in an Envoy proxy at the edge. It loads a compiled descriptor set of your protos, so you do not need to generate or compile Go code.
Teams adopt transcoding so that one contract can serve both gRPC and REST consumers. This removes hand-written handlers that drift from the service definition.
How does it work?
In short, the transcoder needs a machine-readable copy of your service definition and a routing table built from it. grpc-gateway gets both at build time: the plugin turns each annotation into generated Go routing and marshalling code, compiled into the proxy binary. Envoy gets both at load time: it parses the binary descriptor set, builds its routing table from the google.api.http rules it finds, and uses the message descriptors to convert JSON to protobuf and back at runtime. In both cases the backend only ever receives ordinary gRPC calls, so the service code is the same whether a REST layer exists or not. Native gRPC clients can keep calling the backend directly, as shown below.
Option A: grpc-gateway (generated Go reverse proxy)
+-------------+ HTTP/JSON +------------------+ gRPC +-------------+
| REST client |------------>| grpc-gateway |---------->| gRPC |
| |<------------| generated Go |<----------| backend |
+-------------+ JSON | proxy | protobuf | service |
+------------------+ +-------------+
^
+-------------+ |
| gRPC client |---------------------- gRPC (direct) --------------+
+-------------+
Option B: Envoy gRPC-JSON transcoder filter (edge)
+-------------+ HTTP/JSON +------------------+ gRPC +-------------+
| REST client |------------>| Envoy |---------->| gRPC |
| |<------------| gRPC-JSON |<----------| backend |
+-------------+ JSON | transcoder filter| protobuf | service |
+-------------+ | (descriptor set)| +-------------+
| gRPC client |------------>| gRPC passes |
+-------------+ gRPC | through |
+------------------+
Per request, inside either transcoder:
[1 Match] verb + path vs google.api.http annotations
--> [2 Merge] path vars + JSON body + query params -> request message
--> [3 Call] backend over gRPC
--> [4 Serialize] response via canonical proto3 JSON mapping
Two ways to place a transcoder between REST clients and one gRPC backend: a grpc-gateway Go proxy or an Envoy filter at the edge, both following the match, merge, call, serialize lifecycle.
Who should use gRPC REST transcoding?
Transcoding fits teams that already have, or are committed to, gRPC services and that also need to serve callers who cannot or will not speak gRPC. Typical cases:
- Backend teams with gRPC internals and HTTP/JSON consumers. Browser front ends, partner integrations, webhooks, and scripts that use
curlor a generic HTTP library are common examples. - Platform and infrastructure teams running a shared edge. If Envoy already terminates traffic for many services, adding the transcoder filter gives every proto-defined service a REST surface without each team writing handlers.
- Go service teams that want one deployable. grpc-gateway lets them ship the REST proxy in the same binary as the gRPC server.
- Teams migrating from REST to gRPC. Existing REST clients can keep working through annotated routes, including legacy paths via
additional_bindings, while new clients move to gRPC.
It is a poor fit for teams whose services are REST-only with no gRPC plans, or whose APIs depend on streaming patterns and REST conventions that the annotations cannot express. The last section covers these cases in more detail.
How do you write google.api.http annotations correctly?
Each RPC carries a google.api.http option, and that option is the whole REST contract. Pick exactly one verb (get, post, put, patch, or delete) and give it a path template. Follow resource-oriented conventions: GetBook maps to get, CreateBook to post, and UpdateBook to patch when it takes a field mask. Proxies and caches treat GET as safe, so a GET must never mutate state.
A template such as /v1/{name=projects/*/books/*} binds the request's name field to a multi-segment path. The value keeps its slashes, so name arrives as projects/p1/books/b7. GET and DELETE take no body. For writes, body: "*" maps every field not bound to the path from the JSON body. body: "book" maps only the book field from the body. Use the named form for update calls, because it leaves sibling fields like update_mask available as query parameters.
When a field is bound to neither the path nor the body, it becomes a query parameter. Nested message fields use dotted names (?filter.author=ada). Repeated fields repeat the key (?tags=a&tags=b). Map fields cannot be set this way. Under body: "*" nothing is left over, so there are no query parameters at all. This is a reason to keep list and filter requests flat and built from scalar fields.
Use additional_bindings to add alternate routes, such as a legacy path or a second parent collection. Bindings cannot be nested inside one another.
Treat these annotations as published API. If you skip them or mirror RPC names (/v1/BookService/GetBook), gRPC internals leak into URLs. HTTP clients have no generated stubs to catch drift. Renaming a field bound to a path or query parameter, or moving it into the body, breaks them even though the change is wire-compatible for gRPC clients.
syntax = "proto3";
package library.v1;
import "google/api/annotations.proto";
import "google/protobuf/field_mask.proto";
service BookService {
// name is bound to the path and keeps its slashes: projects/p1/books/b7
rpc GetBook(GetBookRequest) returns (Book) {
option (google.api.http) = { get: "/v1/{name=projects/*/books/*}" };
}
// Unbound fields become query params:
// GET /v1/projects/p1/books?page_size=20&filter.author=ada&tags=a&tags=b
rpc ListBooks(ListBooksRequest) returns (ListBooksResponse) {
option (google.api.http) = {
get: "/v1/{parent=projects/*}/books"
// Alternate route for a second parent collection (cannot be nested)
additional_bindings { get: "/v1/{parent=projects/*/shelves/*}/books" }
};
}
// body: "book" leaves update_mask free to arrive as ?update_mask=title,author
rpc UpdateBook(UpdateBookRequest) returns (Book) {
option (google.api.http) = {
patch: "/v1/{book.name=projects/*/books/*}"
body: "book"
};
}
}
message Filter { string author = 1; }
message GetBookRequest { string name = 1; }
message ListBooksRequest {
string parent = 1;
int32 page_size = 2;
Filter filter = 3; // dotted form: ?filter.author=ada
repeated string tags = 4; // repeated key: ?tags=a&tags=b
}
message UpdateBookRequest {
Book book = 1;
google.protobuf.FieldMask update_mask = 2;
}
message Book { string name = 1; string title = 2; string author = 3; }
message ListBooksResponse { repeated Book books = 1; }
A proto service whose google.api.http annotations show a multi-segment path variable, query parameters derived from nested and repeated fields, an additional_bindings route, and body: "book" leaving update_mask available as a query parameter.
How do you generate and run a grpc-gateway reverse proxy?
Generating the gateway code
The grpc-gateway plugin, protoc-gen-grpc-gateway, runs alongside the standard Go and Go-gRPC plugins. It reads your google.api.http annotations and emits a *.pb.gw.go file per proto file. With buf, add the plugin as a third entry in buf.gen.yaml next to the other two, then run buf generate. With raw protoc, pass --grpc-gateway_out and make sure google/api/annotations.proto and http.proto are on the include path. Buf resolves these through a dependency, which is the main reason teams prefer it here.
Wiring the server
The generated file exposes a Register<Service>HandlerFromEndpoint function. You create a runtime.ServeMux, register each service against the backend's gRPC address, and serve that mux from a normal net/http server. Each incoming request is decoded from JSON into the request message and sent over a real gRPC client connection.
Headers and middleware
Header forwarding is where most surprises happen. By default, the mux forwards standard HTTP headers as gRPC metadata with a grpcgateway- prefix, so the backend sees grpcgateway-authorization instead of authorization. Install a custom header matcher with runtime.WithIncomingHeaderMatcher to pass authorization, traceparent, or your own x- headers through under their own names. Use runtime.WithMetadata to derive metadata from the request, such as a verified principal. Put HTTP-level concerns like CORS, request logging, and token pre-validation in ordinary http.Handler middleware that wraps the mux.
Operational cost
You now run and deploy an extra Go process. Every proto change also requires regenerating code and redeploying the gateway as well as the service.
version: v2
modules:
- path: proto
deps:
- buf.build/googleapis/googleapis
---
version: v2
plugins:
# Message types (*.pb.go)
- remote: buf.build/protocolbuffers/go
out: gen/go
opt: paths=source_relative
# gRPC client and server stubs (*_grpc.pb.go)
- remote: buf.build/grpc/go
out: gen/go
opt: paths=source_relative
# REST reverse proxy (*.pb.gw.go). Reads google.api.http
# annotations and emits Register<Service>HandlerFromEndpoint.
- remote: buf.build/grpc-ecosystem/gateway
out: gen/go
opt:
- paths=source_relative
# Only expose RPCs that have an explicit HTTP annotation.
- generate_unbound_methods=false
*A buf v2 setup that pulls in the googleapis annotations as a dependency and runs the Go, Go-gRPC, and grpc-gateway plugins together so each proto produces a .pb.gw.go handler alongside the usual stubs.
How do you configure Envoy's gRPC-JSON transcoder?
Envoy's transcoder does at the edge what grpc-gateway does in a separate process. It needs no generated Go code, but it needs one artifact: a binary descriptor set.
Build the descriptor set
Run protoc with --include_imports and --descriptor_set_out. The --include_imports flag matters. Without it, the descriptor lacks google/api/annotations.proto, http.proto and any well-known types you reference. Envoy then fails to resolve the HTTP rules or message types and rejects the filter config at load time.
Configure the filter and route
Add the grpc_json_transcoder HTTP filter. Point it at the descriptor file, and list each service by its fully qualified name, such as bookstore.v1.BookService. Only listed services are transcoded. Route matched traffic to a cluster that has HTTP/2 enabled. gRPC requires HTTP/2, so an HTTP/1.1 upstream connection fails even when transcoding succeeds.
Headers
Envoy forwards incoming HTTP headers upstream, and the gRPC server sees them as metadata. Strip or overwrite headers you don't want clients to forge, such as internal identity headers, before they reach the filter. Do this with route- or listener-level header manipulation.
The stale-descriptor failure
Envoy reads the descriptor at config load, and nothing ties it to your deployed services. Suppose you add an RPC and deploy the server without updating the descriptor. Requests to the new path match no binding. Envoy passes them through untransformed, and they surface as unexplained 404s. Renamed or renumbered fields produce silent JSON mismatches instead. You can enable the filter's request validation options to reject unknown methods explicitly, which turns these cases into clearer errors. The structural fix is to generate the descriptor in the same pipeline step that compiles the protos and builds the server. Ship the descriptor with the server so the two versions cannot drift.
static_resources:
listeners:
- name: rest_edge
address: { socket_address: { address: 0.0.0.0, port_value: 8080 } }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: rest_edge
route_config:
# Drop client-supplied identity headers so they never reach the gRPC server as metadata
request_headers_to_remove: ["x-internal-user-id"]
virtual_hosts:
- name: bookstore
domains: ["*"]
routes:
- { match: { prefix: "/" }, route: { cluster: bookstore_grpc, timeout: 15s } }
http_filters:
- name: envoy.filters.http.grpc_json_transcoder
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.grpc_json_transcoder.v3.GrpcJsonTranscoder
proto_descriptor: /etc/envoy/bookstore.pb
services: ["bookstore.v1.BookService"] # only listed services are transcoded
# Turn stale-descriptor pass-through 404s into explicit rejections
request_validation_options: { reject_unknown_method: true }
- name: envoy.filters.http.router
typed_config: { "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router }
clusters:
- name: bookstore_grpc
type: STRICT_DNS
# gRPC needs HTTP/2 to the upstream, even though clients speak HTTP/1.1 JSON
typed_extension_protocol_options:
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
"@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
explicit_http_config: { http2_protocol_options: {} }
load_assignment:
cluster_name: bookstore_grpc
endpoints: [{ lb_endpoints: [{ endpoint: { address: { socket_address: { address: bookstore, port_value: 9090 } } } }] }]
A minimal Envoy v3 config that loads a descriptor set built with --include_imports, transcodes one fully qualified service, strips a forgeable identity header, rejects unknown methods so a stale descriptor fails loudly, and sends traffic to an HTTP/2 gRPC cluster.
How are errors and JSON serialization handled for REST clients?
grpc-gateway translates a gRPC status into an HTTP status using a fixed table. NotFound becomes 404, InvalidArgument becomes 400, Unauthenticated becomes 401, PermissionDenied becomes 403, Unavailable becomes 503, and Internal and Unknown both become 500. By default the response body is the JSON form of google.rpc.Status, with code, message and a details array of Any messages tagged with @type.
This translation loses information. Several gRPC codes collapse into one HTTP code, so FailedPrecondition and InvalidArgument both arrive as 400. The details entries are useful only to clients that know the referenced proto types. A browser client receiving type.googleapis.com/google.rpc.BadRequest cannot do much with it. If you need predictable errors, register a custom handler with runtime.WithErrorHandler. That handler can produce one stable shape, such as {error, message, fields}, and keep internal messages out of responses. With Envoy, enable convert_grpc_status so the status is rendered as a JSON body instead of being left in trailers.
Proto3 JSON mapping also surprises clients. The table shows the defaults and the options that change them.
| Behavior | Default | grpc-gateway (protojson.MarshalOptions) |
Envoy print_options |
|---|---|---|---|
| Field names | lowerCamelCase | UseProtoNames |
preserve_proto_field_names |
| Zero values | omitted | EmitUnpopulated |
always_print_primitive_fields |
| Enums | name string | UseEnumNumbers |
always_print_enums_as_ints |
| int64/uint64 | quoted string | no option | no option |
The int64-as-string rule is deliberate and has no off switch. JavaScript numbers lose precision above 2^53, so quoting large integers keeps them intact. Timestamp is rendered as an RFC 3339 string in UTC, such as "2024-01-01T00:00:00Z". Duration is rendered as a seconds string, such as "1.5s". Document these formats explicitly, because clients will otherwise try to parse them as numbers.
How do you document and test both the REST and gRPC paths?
The fix for that guessing is to generate the REST documentation from the same google.api.http annotations the proxy uses. grpc-gateway ships protoc-gen-openapiv2, a protoc plugin that emits an OpenAPI v2 spec with paths, methods, path and query parameters, and request bodies derived from your proto. Run it in the same build step as the gateway code so that a single proto change regenerates the stubs, the proxy and the spec together. A hand-maintained spec drifts quietly, because nothing fails when it is wrong. A generated spec can be checked in CI: regenerate it and fail the build if the diff is non-empty.
Testing both paths against one backend
Run one backend instance behind both entry points. Then call each method twice: once over REST with curl, and once over gRPC with a CLI client or a small generated client. Compare three things:
- Responses: the JSON body should match the protobuf message field for field, including default-value handling and the well-known type formats documented above.
- Errors: a
NOT_FOUNDfrom the service should arrive as gRPC status 5 on the native path and as HTTP 404 with the same message on the REST path. - Metadata: send an
Authorizationheader and atraceparentheader over REST, and confirm the service sees them as gRPC metadata.
Keep the tests that exercise auth, rate limiting and tracing. A policy enforced only in a gRPC interceptor, or only at the HTTP edge, leaves the other entry point open. Enforce each policy where both paths converge, or test explicitly that both paths enforce it identically.
What are the alternatives to gRPC REST transcoding?
If you need both gRPC and HTTP/JSON access, transcoding is one option among several. Each alternative moves the cost to a different place.
- Hand-written REST handlers. You write HTTP handlers that call the gRPC service or the same business logic directly. This gives you full control over URLs, status codes, caching headers, and JSON shape. The cost is a second contract: every proto change must be mirrored by hand in the handlers, the spec, and the tests.
- gRPC-Web. Browser clients speak a gRPC variant over HTTP/1.1 or HTTP/2 through a compatible proxy, using generated client stubs. You keep protobuf types end to end, and server streaming is supported. However, clients need a generated library, and the API is not usable from plain
curlin any practical way. - Protocols that serve gRPC and plain HTTP from one handler. Some RPC frameworks accept gRPC, gRPC-Web, and a simple HTTP POST with a JSON body on the same endpoint. This removes the separate proxy, but URLs follow RPC names rather than resource paths, so it does not produce a resource-oriented REST API.
- OpenAPI-first design. You define the API in an OpenAPI spec and generate server and client code from it, with gRPC used only internally if at all. This suits public APIs designed around HTTP semantics, but it gives up the proto file as the single source of truth.
- Native gRPC only. You skip the HTTP surface entirely and require every consumer to use gRPC. This is the simplest option when all callers are internal services you control.
The comparison table in the final section summarizes the trade-offs.
How can transcoding save engineering time?
The time savings come from removing work that would otherwise be repeated for every RPC and every change:
- No hand-written handlers. Path parsing, query parameter decoding, JSON marshalling, and request-message assembly are generated or handled by the proxy. You do not write them per endpoint.
- One change, one place. Adding a field or an RPC means editing the proto and its annotation. Codegen or the descriptor build updates the REST surface, and
protoc-gen-openapiv2updates the spec in the same step. - Drift caught by tooling. A CI check that regenerates code and the OpenAPI spec and fails on a non-empty diff replaces manual review of whether REST and gRPC still agree.
- Consistent behavior across services. Error mapping, header forwarding, and JSON conventions are configured once in the gateway or Envoy, instead of being re-implemented, and re-debugged, in each team's handlers.
An illustrative estimate, using assumed inputs rather than measured data: assume a service with 30 RPCs, and assume a hand-written REST handler with tests takes about 2 hours per RPC to build and about 30 minutes per later change. Building the REST layer by hand would then take about 60 hours. If the team makes an assumed 10 proto changes per month that touch the REST surface, that adds about 5 hours per month of mirroring work. With transcoding, assume each RPC needs about 10 minutes to write and review its annotation, or about 5 hours in total, plus a one-time setup of the proxy, CI codegen, and the descriptor or gateway build, assumed at 1 to 2 days. Under these assumptions, transcoding is cheaper after the first service, and the monthly mirroring work drops to near zero. Your numbers will differ. Recompute with your own RPC count, change rate, and handler complexity, and include the ongoing cost of operating the proxy.
The savings shrink when you need many custom error handlers, header rules, or REST-specific behaviors, because that work moves into gateway middleware or Envoy configuration rather than disappearing.
When should you use grpc-gateway, Envoy, or avoid transcoding altogether?
Choose grpc-gateway when your team writes Go and wants transcoding to ship inside its own binary or sidecar. The proxy is ordinary Go code that you version, test and deploy with the service, and you can add middleware in the same language. Choose Envoy when services are written in several languages, or when Envoy already sits in front of them. In that case transcoding becomes configuration, and application code stays the same.
| Option | Best fit | Main cost |
|---|---|---|
| grpc-gateway | Go teams, self-contained deployables | Extra codegen step, Go-only |
| Envoy transcoder | Polyglot teams, existing Envoy | Descriptor distribution, proxy config |
| Hand-written REST | Public, resource-oriented APIs | Two contracts to keep in sync |
| gRPC-Web / Connect-style | Browser clients needing streaming | Client libraries required |
| Native gRPC only | Internal service-to-service | No plain-HTTP access |
Avoid transcoding in the following cases:
- Client or bidirectional streaming is central to the API. These patterns have no clean request/response HTTP equivalent.
- You are building a public REST API. It needs idiomatic resources, pagination and caching semantics that RPC-shaped annotations express poorly.
- The path is hot. The JSON-to-protobuf hop adds CPU and latency, so measure it under your own load before committing.
- Clients reject proto3 JSON conventions. Examples include int64 values encoded as strings and omitted default values.
The benefit of transcoding is one contract, from which you generate handlers and an OpenAPI spec. The cost is extra build steps, proxy configuration, and two surfaces to secure and observe. Accept that cost only if the second surface has real consumers.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to grpc rest transcoding, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.