Spring Boot gRPC Streaming: Server, Client and Bidirectional

What gRPC streaming is and how the four RPC types map to Java

A streaming RPC keeps one HTTP/2 stream open and sends a sequence of messages over it, rather than one request and one response. In the .proto file, the stream keyword before the request type, the response type, or both selects one of four RPC types. That keyword also determines the Java method signature that grpc-java generates for your service base class.

Unary and server streaming have the same Java signature. The only difference is how many times you call onNext before onCompleted. Client and bidirectional streaming invert the shape: you return a StreamObserver<Req>, and gRPC pushes incoming messages into it.

Treat each type as a contract with three parts:

• Pace: who decides when the next message is sent.
• Lifecycle: who decides the call is finished.
• Errors: who can terminate the call with onError while the other side is still sending.

Teams choose streaming for four main reasons:

• Incremental results: the client can render partial search output before the full result set is ready.
• Long-running feeds: price ticks or change events arrive over one connection, with no polling.
• Uploads: the server receives chunks as they are sent and does not need the whole payload in memory.
• Interactive sessions: both sides exchange messages with low latency.

Copy// demo.proto (Req and Resp each have a string field `text`):
//   service DemoService {
//     rpc Get(Req) returns (Resp);
//     rpc Watch(Req) returns (stream Resp);
//     rpc Upload(stream Req) returns (Resp);
//     rpc Chat(stream Req) returns (stream Resp);
//   }
import io.grpc.stub.StreamObserver;

public class DemoServiceImpl extends DemoServiceGrpc.DemoServiceImplBase {
  // Unary: one onNext, then onCompleted.
  @Override public void get(Req req, StreamObserver<Resp> out) {
    out.onNext(Resp.newBuilder().setText("one").build());
    out.onCompleted();
  }
  // Server streaming: same signature. The server picks how many messages to send and when to stop.
  @Override public void watch(Req req, StreamObserver<Resp> out) {
    for (int i = 0; i < 3; i++) out.onNext(Resp.newBuilder().setText("update " + i).build());
    out.onCompleted();
  }
  // Client streaming: return an observer. The client's onCompleted ends the input, then the server answers once.
  @Override public StreamObserver<Req> upload(StreamObserver<Resp> out) {
    return new StreamObserver<Req>() {
      int count;
      public void onNext(Req r) { count++; }
      public void onError(Throwable t) { /* client failed the call; do not respond */ }
      public void onCompleted() {
        out.onNext(Resp.newBuilder().setText("received " + count).build());
        out.onCompleted();
      }
    };
  }
  // Bidirectional: each side sends independently. This server replies to every message as it arrives.
  @Override public StreamObserver<Req> chat(StreamObserver<Resp> out) {
    return new StreamObserver<Req>() {
      public void onNext(Req r) { out.onNext(Resp.newBuilder().setText("echo " + r.getText()).build()); }
      public void onError(Throwable t) { /* stream already terminated by the client */ }
      public void onCompleted() { out.onCompleted(); }
    };
  }
}

A grpc-java service that implements all four RPC types. Unary and server streaming share a signature that takes a StreamObserver, while client and bidirectional streaming return a StreamObserver that gRPC feeds with incoming messages.

How gRPC streaming works

Every gRPC call, streaming or not, is one HTTP/2 stream. Many such streams are multiplexed over a single TCP connection, so a long-lived stream does not block other calls on the same channel.

A call proceeds in these steps:

1. Call start: the client sends a HEADERS frame. It carries the method path (/package.Service/Method), the content type, any deadline as a grpc-timeout header, and your custom metadata.
2. Messages: each protobuf message is serialized and framed with a short prefix. The prefix holds a compression flag and the message length. The framed messages travel in HTTP/2 DATA frames. A large message can span several frames, and several small messages can share one. The receiver reassembles them by length, so message boundaries do not depend on frame boundaries.
3. Half-close: when the client has no more messages, it sets END_STREAM on its last frame. In grpc-java this is what calling onCompleted on the request observer does. The server can keep sending after the client half-closes.
4. Completion: the server ends the call by sending trailers that carry grpc-status and, optionally, grpc-message. Calling onCompleted on the server sends an OK status. Calling onError sends the code from your Status. A status is always final: no messages follow it.
5. Cancellation: either side can abort with an HTTP/2 RST_STREAM. This is how client cancellation and deadline expiry reach the server.

The four RPC types differ only in how many messages each side sends between those steps. The wire protocol is the same. That is why unary and server streaming share a Java signature.

In grpc-java, the generated stubs hide this framing behind StreamObserver. Outbound, each onNext serializes one message and queues it for the transport. Inbound, the transport decodes each message and invokes your observer's onNext on an executor thread. HTTP/2 flow-control windows limit how many bytes the sender can have unacknowledged. grpc-java also exposes a message-level readiness signal (isReady() and the on-ready handler) and explicit request(n) calls. The flow-control section below covers these.

Who should use gRPC streaming

Streaming suits Spring Boot teams that already use gRPC or protobuf between services and have at least one of these needs:

• Backend teams publishing event or change feeds to other services. Server streaming replaces polling loops and avoids a separate pub/sub client for point-to-point delivery.
• Teams moving large or open-ended payloads, such as file uploads, bulk imports or report exports. Client or server streaming sends chunks instead of one oversized message, which keeps memory bounded on both ends.
• Platform and infrastructure teams building agents, telemetry collectors or control channels. Here a component holds one connection and exchanges messages in both directions.
• Teams building interactive features behind a backend-for-frontend, such as chat, collaborative editing or live dashboards. This works where every hop supports the required streaming mode.

Streaming is a poor fit in these cases:

• Browser clients that call your service directly and need client or bidirectional streaming. gRPC-Web does not support those modes.
• Bounded, small result sets. Paginated unary RPCs are simpler.
• Teams without capacity to own the operational concerns. These include backpressure, cancellation cleanup, keepalive and connection recycling, all covered below. Without them, a streaming service tends to fail under load, not in tests.

Wiring streaming services and clients into Spring Boot

Every generated *ImplBase implements BindableService, and that is what the integration looks for. Annotate your subclass as a Spring bean and let the integration register every BindableService in the context with the server it manages. This keeps the wiring the same across integrations and avoids starter-specific properties. On the client side, declare a ManagedChannel as a bean. Use its shutdown method as the destroy method so that in-flight calls drain when the context closes. Derive async stubs from that channel. Stubs are cheap and safe to share, but channels are not cheap, so create one channel per target.

Copy+------------------------- Spring ApplicationContext -------------------------+
|                                                                             |
|  [gRPC Server] <-- registers -- [OrderStreamService bean (ImplBase)]        |
|       ^                         [TelemetryService bean   (ImplBase)]        |
|       |  ServerInterceptor beans (auth, MDC -> Context)                     |
|                                                                             |
|  [ManagedChannel bean] --(destroy: shutdown)--> [async stub beans]          |
+-----------------------------------------------------------------------------+

Where Spring conventions stop working

A stream outlives the method that opens it. Inbound messages arrive on gRPC executor threads through StreamObserver callbacks. This breaks three common Spring assumptions:

• Request scope has no active request on those threads, so request-scoped beans fail or resolve to the wrong instance.
• @Transactional commits when the service method returns, which happens before the stream ends. Open a short transaction per message, not one per stream.
• Thread-local state such as SecurityContextHolder and MDC is empty on callback threads.

Use a ServerInterceptor to carry this state instead. It authenticates once, stores the principal and correlation ID under Context.Keys, and attaches the resulting context with Contexts.interceptCall. gRPC restores that Context for every listener callback. If you need MDC for logging, populate it from the Context inside each callback.

Copy@Configuration
public class GrpcClientConfig {
  // One channel per target. Spring calls shutdown() on context close, so in-flight calls drain.
  @Bean(destroyMethod = "shutdown")
  ManagedChannel orderChannel() {
    return ManagedChannelBuilder.forAddress("orders.internal.example", 9090)
        .usePlaintext() // placeholder only; configure TLS for real deployments
        .build();
  }

  // Stubs are cheap and thread-safe. Share one per channel.
  @Bean
  OrderStreamGrpc.OrderStreamStub orderStub(ManagedChannel orderChannel) {
    return OrderStreamGrpc.newStub(orderChannel);
  }
}

// Moves the request id into gRPC Context instead of a ThreadLocal, so it
// survives the hop to whichever executor thread runs each StreamObserver callback.
@Component
class RequestIdInterceptor implements ServerInterceptor {
  static final Context.Key<String> REQUEST_ID = Context.key("request-id");
  private static final Metadata.Key<String> HEADER =
      Metadata.Key.of("x-request-id", Metadata.ASCII_STRING_MARSHALLER);

  @Override
  public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(
      ServerCall<ReqT, RespT> call, Metadata headers, ServerCallHandler<ReqT, RespT> next) {
    String id = headers.get(HEADER);
    Context ctx = Context.current().withValue(REQUEST_ID, id != null ? id : "unknown");
    // Attaches ctx around every listener callback (onMessage, onHalfClose, onCancel).
    // Service code reads it with RequestIdInterceptor.REQUEST_ID.get().
    return Contexts.interceptCall(ctx, call, headers, next);
  }
}

A ManagedChannel bean with shutdown as its destroy method and a shared async stub, plus a ServerInterceptor that puts a request id in gRPC Context so it is available in every stream callback.

Implementing server, client and bidirectional streaming

Server streaming

A server-streaming method receives one request and a StreamObserver for responses. Emit each event with onNext and end the stream with onCompleted. Do not run the emit loop inside the service method. That method runs on the gRPC executor, and a loop that waits on a queue or an external feed holds the executor thread for as long as the stream is open. Hand the work to your own executor and return right away. Cast the observer to ServerCallStreamObserver so you can check isCancelled() between events. Wrap the loop so that every exception ends in onError, and make no further calls on the observer after it. A stream that is never closed holds its resources until the client's deadline expires.

Client streaming

In client streaming, the service method returns a StreamObserver that the client writes into. Keep running state in that observer, such as a byte count, a checksum or a partial aggregate, and update it on each onNext. Send the single response in onCompleted, which is the only point where the input is known to be complete. On the client side, the async stub gives you a request observer. You call onNext once per chunk and then onCompleted, and the reply arrives on the response observer you passed in.

Bidirectional streaming

In a chat or command channel, each direction has its own lifecycle. The client can half-close while the server keeps sending, and the reverse is also allowed. StreamObserver is not thread-safe. If a thread pool pushes replies without coordination, the calls can interleave. This usually passes in tests and then fails intermittently under load. Send all outbound calls through one lock or a single-threaded executor per call, so that only one thread ever calls onNext, onError or onCompleted.

Copy// Bidirectional chat: replies are computed on a shared pool, but every call on `out` is serialized.
private final ExecutorService workers = Executors.newFixedThreadPool(8);

@Override
public StreamObserver<ChatMessage> chat(StreamObserver<ChatMessage> out) {
  return new Session(out);
}

private final class Session implements StreamObserver<ChatMessage> {
  private final StreamObserver<ChatMessage> out;
  // Guarded by `this`: these fields and EVERY call on `out` (onNext/onError/onCompleted).
  private int inFlight;
  private boolean clientHalfClosed, closed;

  Session(StreamObserver<ChatMessage> out) { this.out = out; }

  @Override public void onNext(ChatMessage msg) {
    synchronized (this) { inFlight++; }
    workers.execute(() -> {          // `this` in the lambda is still the Session
      ChatMessage reply;
      try { reply = handle(msg); } catch (RuntimeException e) { fail(e); return; }
      synchronized (this) {          // only one thread touches `out` at a time
        inFlight--;
        if (closed) return;          // nothing may follow onError/onCompleted
        out.onNext(reply);
        maybeComplete();
      }
    });
  }

  // Client failed or cancelled: stop sending, never call `out` again.
  @Override public synchronized void onError(Throwable t) { closed = true; }

  // Client half-closed its side; the server direction stays open until pending replies are sent.
  @Override public synchronized void onCompleted() { clientHalfClosed = true; maybeComplete(); }

  private synchronized void fail(Throwable t) {
    if (!closed) { closed = true; out.onError(Status.INTERNAL.withCause(t).asRuntimeException()); }
  }

  private void maybeComplete() {     // caller holds the lock
    if (clientHalfClosed && inFlight == 0 && !closed) { closed = true; out.onCompleted(); }
  }
}

A bidirectional streaming handler that processes messages on a thread pool. All outbound StreamObserver calls go through one per-call lock, and the server finishes its own direction only after the client has half-closed and every pending reply has been sent.

Flow control, cancellation and deadlines

HTTP/2 flow control limits how many bytes are in flight on the connection, but it does not limit how much your application queues. When the peer's window is full, gRPC-Java accepts each onNext call and buffers the message in memory. A loop that streams a large result set behind a slow consumer will pass every test against a fast local client. In production it fills the heap until the service runs out of memory. This is the most common way streaming services fail.

The fix is to send only when the transport can accept more. On the server, cast the StreamObserver to ServerCallStreamObserver. On the client, implement ClientResponseObserver and capture the ClientCallStreamObserver in beforeStart. Then register setOnReadyHandler() and, inside the handler, emit messages only while isReady() returns true. When it returns false, stop and wait for the next callback.

Inbound flow control works the same way in reverse. Call disableAutoRequest() on the server, or disableAutoRequestWithInitial(n) on the client. After that, call request(n) only when your processing has capacity for more messages.

Set deadlines with withDeadlineAfter on the stub. The deadline is absolute and is computed when you call that method, so derive the stub per call rather than storing it in a bean. On long-lived streams, the deadline applies to the entire stream, not to each message.

To detect client cancellation, use setOnCancelHandler() or a listener on Context.current(). In that handler, close database cursors, stop message consumers and cancel scheduled work. Otherwise each abandoned stream keeps its upstream resources open indefinitely.

Copy@Override
public void streamOrders(OrderQuery request, StreamObserver<Order> responseObserver) {
  ServerCallStreamObserver<Order> observer =
      (ServerCallStreamObserver<Order>) responseObserver;
  // Lazily fetched result set. It must be closed on every exit path.
  OrderCursor cursor = orderRepository.openCursor(request.getCustomerId());
  AtomicBoolean done = new AtomicBoolean(false);

  // Runs when the client cancels or the deadline expires. Release upstream resources.
  observer.setOnCancelHandler(() -> {
    if (done.compareAndSet(false, true)) {
      cursor.close();
    }
  });

  // gRPC invokes this whenever the transport can accept more data.
  observer.setOnReadyHandler(() -> {
    try {
      // Write only while the outbound buffer has room. Never loop blindly.
      while (observer.isReady() && !done.get()) {
        if (cursor.hasNext()) {
          observer.onNext(cursor.next());
        } else if (done.compareAndSet(false, true)) {
          cursor.close();
          observer.onCompleted();
        }
      }
      // isReady() is false: return and wait for the next onReady callback.
    } catch (RuntimeException e) {
      if (done.compareAndSet(false, true)) {
        cursor.close();
        observer.onError(Status.INTERNAL.withCause(e).asRuntimeException());
      }
    }
  });
}

// Client side: derive a fresh stub per call so the absolute deadline starts now.
// stub.withDeadlineAfter(30, TimeUnit.SECONDS).streamOrders(query, responseObserver);

This server-streaming handler writes only while isReady() is true, resumes from setOnReadyHandler, and closes its database cursor on completion, error or client cancellation. The client sets a per-call deadline on a freshly derived stub.

Error handling, status codes and testing streaming endpoints

Report failures by calling onError with a StatusRuntimeException built from an explicit code. Use Status.INVALID_ARGUMENT for a malformed request message, UNAVAILABLE when a downstream dependency is unreachable, and DEADLINE_EXCEEDED when your own upstream budget runs out. Pass the exception to onError and do not call onNext afterward.

Throwing from a request observer's onNext does not do the same job. grpc-java catches the exception and closes the call with UNKNOWN. Your description is lost, and the client cannot tell a bad input from a bug. To avoid repeating try/catch in every handler, centralize the mapping:

• Server interceptor: wrap the ServerCall.Listener, catch exceptions in onMessage and onHalfClose, and call call.close(status, trailers).
• Spring-level translation: register a bean that maps domain exception types to Status, so services throw domain exceptions and never touch gRPC types.

Mid-stream failures are harder than unary failures. When a stream breaks after a hundred messages, both sides hold partial state. The server may have committed some writes, and the client has processed some responses. gRPC's transparent retry and retry policies stop applying once the response has been committed, so the library will not replay the call for you. You must design resume semantics explicitly. Common options are sequence numbers, resume tokens, or idempotency keys on each message.

Testing streaming endpoints

Use InProcessServerBuilder and InProcessChannelBuilder with a shared unique name and directExecutor(). This exercises the real gRPC call path without opening a socket.

• Ordering: collect responses into a list and assert the sequence after onCompleted.
• Cancellation: cancel from the client through ClientCallStreamObserver.cancel or a cancellable Context. Then await a latch that the server's setOnCancelHandler counts down, which proves the cleanup ran.

Running long-lived streams in production

A stream that is open but silent looks the same as one whose peer has disappeared. Load balancers and NAT gateways drop idle TCP connections after a timeout, often without telling either side. Configure HTTP/2 keepalive pings on the client with keepAliveTime on the channel builder. The server must also allow them through permitKeepAliveTime, or it will close the connection with GOAWAY and too_many_pings. Keepalive only proves that the transport is alive. An application-level heartbeat message, sent every N seconds and checked against a deadline, also catches a producer that is stuck while its connection stays healthy.

Behind an L4 load balancer, a channel pins to one backend for its whole lifetime. New instances that you add during a scale-out receive no traffic until clients reconnect.

A ServerInterceptor runs once per call, which suits auth and tagging the call with metadata. Per-message logic, such as counting messages or validating payloads, belongs in a ForwardingServerCallListener (onMessage) and a ForwardingServerCall (sendMessage).

Watch these metrics:

• Active streams: shows imbalance between instances and leaks.
• Messages per stream: reveals streams that are stalled or running away.
• Stream duration: checks that recycling actually happens.

Check every hop before you choose a streaming mode. gRPC-Web supports unary and server streaming only, so browsers cannot use it for client or bidirectional streaming. Some gateways and proxies also buffer messages or reject long-lived bidirectional streams.

When to avoid streaming, alternatives, and impact on engineering cost

Those limits hit some streaming modes harder than others. Streaming pays off for teams whose data is unbounded or arrives incrementally: change feeds, log tails, telemetry, or interactive sessions where each side reacts to the other's messages. For small, bounded result sets, a paginated unary RPC is usually the better fit. Each page is an independent call, so retries, deadlines, load balancing and request logging work as they do for every other endpoint. A failed page is simply re-requested, with no reconnect-and-resume logic.

Many designs labelled "bidirectional" only need server streaming, with occasional unary calls for client commands. That split keeps each RPC simpler to test.

Illustrative estimate (assumed numbers, not a benchmark):

• Formula: total hours = harness + endpoints × (paths × hours per path + monitoring).
• Without a shared harness: assume 4 extra paths (cancellation, deadline, backpressure, reconnect) at 3 hours each, plus 2 hours of monitoring. One endpoint costs 4 × 3 + 2 = 14 hours, so five endpoints cost 70 hours.
• With a shared harness: a reusable flow-control, error-mapping and test harness costs 20 hours to build. Assume it cuts per-endpoint work to 4 hours. Five endpoints then cost 20 + 5 × 4 = 40 hours.

How gRPC streaming can save engineering time

Streaming saves time when it replaces infrastructure you would otherwise build by hand, and when the hard parts are solved once rather than per endpoint.

• No hand-rolled push protocol. Without streaming, pushing updates usually means polling endpoints with change tokens, or a WebSocket channel with your own message envelope, framing and error conventions. A stream keyword in the .proto file gives you typed messages, framing, deadlines, cancellation and status codes from the library.
• One contract, generated clients. The .proto file is the API definition, and protoc generates the stubs for Java and other supported languages. Teams do not hand-write client libraries or keep separate API documents in sync with the message format.
• Fewer moving parts for chunked transfer. Client streaming handles uploads in one RPC. The alternative is often a multipart endpoint plus custom chunk-reassembly and progress tracking.
• Solve the hard paths once. Backpressure, cancellation cleanup, error mapping and keepalive are the same for every streaming endpoint. Put them in shared components: an on-ready emitter like the cursor example above, an error-mapping interceptor, a context-propagation interceptor and standard channel settings. New endpoints then reuse that code instead of re-deriving it. The illustrative estimate above shows how this changes total effort under its assumed inputs.
• Fast, socket-free tests. In-process servers and channels with directExecutor() let you test ordering, cancellation and error paths deterministically in ordinary unit tests. This shortens debugging cycles compared with tests that need a running server and network.
• Avoiding streaming where it does not fit also saves time. Choosing paginated unary for bounded data skips resume logic, connection recycling and stream-specific monitoring entirely. Splitting a nominally bidirectional design into server streaming plus unary commands removes the concurrency handling shown in the chat example.

The savings depend on reuse. A single streaming endpoint built from scratch costs more than an equivalent unary endpoint. The cost evens out only when several endpoints share the same harness and conventions.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to spring boot grpc streaming, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.