What is server streaming in grpc-js and how does it work?
A server-streaming RPC takes one request from the client and returns a sequence of response messages, followed by a status that ends the call. In grpc-js, your handler receives a ServerWritableStream. It exposes the decoded request, the call metadata, write() for sending messages, and end() for finishing. The client gets a ClientReadableStream, an object-mode Node.js Readable that emits each decoded message as a data event. It also emits status and error events when the call terminates.
The whole call runs over a single HTTP/2 stream. Each message is serialized, given a 5-byte prefix (a compression flag plus a 4-byte length), and sent as DATA frames. HTTP/2 flow control caps how many unacknowledged bytes the sender may have in flight. The receiver grants more credit with WINDOW_UPDATE frames as it consumes data.
Node.js streams add a second buffer on each side. In object mode, highWaterMark counts messages, not bytes. Pressure moves backward through both buffers:
- A slow client stops reading.
- Its
Readablebuffer fills, so grpc-js stops pulling from the HTTP/2 stream. - The client stops granting window, and the server's HTTP/2 writes stall.
- The server-side buffer grows past its
highWaterMark, andwrite()starts returningfalse.
If the handler ignores that return value, nothing blocks it. Messages pile up in server memory instead.
The rest of this article builds on that mechanism. A reliable streaming service does three things:
- It respects
write()backpressure. - It routes deadline expiry, cancellation and errors through one shutdown path.
- It sets a deadline on every client call.
SERVER CLIENT
+-----------------------+ +-----------------------+
| handler(call) | | ClientReadableStream |
| ServerWritableStream | | (object-mode Readable)|
| call.write(msg) | | 'data' per message |
| call.end() | | 'status' / 'error' |
+-----------+-----------+ +-----------^-----------+
| write() -> false when full | read by app
v |
+-----------------------+ +-----------+-----------+
| writable buffer | | readable buffer |
| highWaterMark (msgs) | | highWaterMark (msgs) |
+-----------+-----------+ +-----------^-----------+
| serialize + 5-byte prefix | decode
v |
+-----------------------+ DATA frames +-----------+-----------+
| HTTP/2 stream (send) | ==================> | HTTP/2 stream (recv) |
| bytes in flight <= | | consumes data |
| flow-control window | <================== | |
+-----------------------+ WINDOW_UPDATE +-----------------------+
BACKPRESSURE (slow client), right to left:
[1] client stops reading -> [2] readable buffer fills, grpc-js stops
pulling -> [3] no WINDOW_UPDATE, server HTTP/2 writes stall -> [4] server
buffer passes highWaterMark and write() returns false
A grpc-js server-streaming call: messages flow from the handler's write() through a Node.js buffer onto one HTTP/2 stream, and a slow client's pressure travels back through the readable buffer and flow-control window until write() returns false.
How does it work from definition to final status?
The diagram above shows the transport. This is the lifecycle of a call from the code's point of view.
- Contract. You declare the RPC in a
.protofile. Thestreamkeyword on the response type makes it server-streaming, for examplerpc StreamReadings(ReadingsRequest) returns (stream Reading);. - Loading. In Node.js you either generate code from the
.protofile or load it at runtime with@grpc/proto-loaderand pass the result togrpc.loadPackageDefinition(). Both produce a service definition and a client constructor. - Registration. The server calls
server.addService()with an object that maps each RPC name to a handler function. It then binds to an address withserver.bindAsync(). A server-streaming handler takes a singlecallargument. It returns no value and sends no callback response. - Invocation. The client calls the method with the request and optional metadata and call options such as a deadline. It gets a
ClientReadableStreamback immediately. The HTTP/2 stream opens, request headers and the request message are sent, and the server's handler runs withcall.requestalready decoded. - Streaming. The handler calls
call.write()for each message. grpc-js serializes each one and frames it onto the HTTP/2 stream. The client decodes each message and emits it as adataevent or yields it tofor await...of. - Termination. The call ends in exactly one way. The handler calls
end(), which sends trailers with statusOK. The handler can instead report an error, which sends trailers with a non-OK code anddetails. Or the client cancels, the deadline passes or the connection drops, in which case the server sees the call as cancelled. The client receives a finalstatusevent, and anerrorevent as well if the status is notOK.
The rest of this article covers the points in this lifecycle where production code usually fails. Step 5 is where backpressure matters. Deadlines are set at step 4 and enforced at step 6. Step 6 is also where cleanup has to run.
Who should use gRPC server streaming?
Server streaming is mainly for these engineers:
- Backend engineers who own Node.js services that already talk gRPC to other internal services and need to return data that is large, unbounded or produced over time.
- Platform and infrastructure teams that build internal APIs such as change feeds, job-status channels or log tails, where many services subscribe to the same source.
- Teams that work across several languages. The
.protocontract generates clients for other languages, so a Node.js streaming service can feed consumers that are not written in JavaScript.
It suits teams willing to own the operational details covered below: write-side backpressure, deadlines on every call, a single cleanup path and resumable messages. A team that only needs to return small payloads, or that mainly serves browsers without a gRPC-Web proxy, gets less from it. The next section covers those cases.
When should you use gRPC server streaming, and when should you avoid it?
Server streaming fits backend and platform engineers who build service-to-service RPCs where the response is large, open-ended or arrives over time:
- Large result sets. Exports, scans and paginated queries. The server sends rows as it reads them instead of buffering the full response in memory.
- Change feeds. The server pushes inserts, updates or events from a log as they happen, and the client tracks its position.
- Progress updates. Long jobs report each stage over the same call that started them, so the client does not need to poll.
- Subscription-style RPCs. A client registers interest once and receives messages until it cancels or the deadline expires.
In each case the client can act on the first message before the last one exists.
Server streaming is the wrong tool in these cases:
- Small, bounded responses. If the response fits comfortably in one message, use a unary call. It is simpler to retry, cache and reason about.
- Browser clients with no gRPC-Web proxy layer. Browsers cannot speak native gRPC directly. Without a translating proxy, the stream never reaches them.
- Fan-out to many slow consumers with no per-client policy. If you have not decided what happens when one client falls behind, its buffered messages grow until the process runs short of memory. You need a defined response such as dropping messages, sampling or disconnecting the client.
- CPU-heavy per-message transforms. These include large protobuf encodes, compression or enrichment done inline.
The last case needs explaining. All streams in a grpc-js process share one event loop. When one handler runs a long synchronous serialization step, every other stream in that process waits. No writes are flushed and no drain events fire until that step finishes. A single expensive transform therefore adds latency to unrelated clients and breaks the backpressure signals the other handlers depend on. Move that work to worker threads or a separate service.
How do you handle backpressure on the server and the client?
Server side
call.write() returns false once the stream's internal buffer passes its highWaterMark. A loop that ignores this keeps queuing messages in process memory while HTTP/2 flow control holds them back from a slow client. Awaiting the 'drain' event before the next write stops that growth. Passing a Readable source to stream.pipeline() with the call as the destination does the pausing for you, and it also forwards errors and tears down both streams.
Client side
for await...of over the response stream reads one message only after the loop body finishes. The client stops reading, the HTTP/2 window fills, and the server sees false. If the handler starts slow async work per message and does not await it, the stream drains at full speed. The buffering then moves into pending promises in application memory.
| Pattern | Where messages wait | Memory bound |
|---|---|---|
Ignore write() result |
Server stream buffer | None |
Await 'drain' / pipeline() |
Producer source | highWaterMark |
| Unawaited per-message work | Client promises | None |
| Bounded concurrency (N in flight) | Client, at most N | N messages |
Trade-offs
Backpressure does not remove delay. It moves latency upstream to the producer. A shared pub/sub producer cannot pause for a single client, so it needs a per-client policy: a capped buffer, dropping or conflating messages, or disconnecting slow consumers. Raising highWaterMark or the HTTP/2 window sizes buys throughput at a cost in memory per stream. Illustrative estimate: 2,000 streams × 1 MiB buffered each ≈ 2 GiB of buffer memory.
const { once } = require('node:events');
const { Readable } = require('node:stream');
const { pipeline } = require('node:stream/promises');
// Server: honour write()'s return value and wait for 'drain' when it is false.
async function streamReadings(call) {
for (let seq = 0; seq < 100000; seq++) {
if (!call.write({ seq, value: Math.random() })) {
await once(call, 'drain'); // pause the producer instead of growing the buffer
}
}
call.end();
}
// Server alternative: pipeline() pauses the source, forwards errors, destroys both.
async function streamReadingsPiped(call) {
async function* source() {
for (let seq = 0; seq < 100000; seq++) yield { seq, value: Math.random() };
}
await pipeline(Readable.from(source()), call).catch((err) => console.error('stream failed:', err.message));
}
// Client: for await reads the next message only when the body yields control.
// Allow at most `limit` handlers in flight; past that, stop reading so the
// HTTP/2 window fills and the server's write() starts returning false.
async function consume(client, handle, limit = 8) {
const call = client.streamReadings({});
const inFlight = new Set();
for await (const msg of call) {
const p = handle(msg).finally(() => inFlight.delete(p));
inFlight.add(p);
if (inFlight.size >= limit) {
await Promise.race(inFlight); // a rejection exits the loop and cancels the call
}
}
await Promise.all(inFlight); // finish work started before the stream ended
}
A server handler that awaits 'drain' or uses stream.pipeline() so a slow client pauses the producer, and a client that consumes with for await...of while capping in-flight per-message work at N so buffering stays bounded.
How do you set and propagate deadlines on gRPC streaming calls in Node.js?
A deadline in grpc-js is an absolute point in time, not a duration. On the client, pass it in the call options as a Date built from the current time plus your timeout, for example new Date(Date.now() + 30_000). When it passes, grpc-js cancels the call, the client stream emits an error whose code equals grpc.status.DEADLINE_EXCEEDED, and the server sees the call as cancelled. Handle that code separately from other failures. It means "we gave up waiting", not "the server is broken", so it should usually lead to a retry with backoff or a resume rather than an alert.
Streaming calls need deadlines more than unary calls do. Without one, nothing in gRPC ends the call on its own. Suppose a peer stops sending but leaves the connection open, for example because it is stuck behind a blocked event loop or a half-dead proxy. The stream then holds its buffers, file descriptors and handler state indefinitely.
Budgeting and propagating on the server
call.getDeadline() tells the handler how much time the caller allows. Subtract Date.now() to get the remaining budget. Use it to decide whether to start an expensive step at all. For each downstream gRPC call, set the deadline to the remaining time minus a safety margin, such as 50–200 ms for network and serialization. Downstream work then stops when your caller gives up, instead of running on for a client that has already left.
Subscription RPCs
Long-lived subscriptions force a trade-off. A short deadline frees resources quickly but cuts off legitimate long streams. There are two common approaches:
- Long deadline plus resume token. The server includes a cursor or sequence number with each message. After any disconnect, the client reconnects and continues from the last token it received. The cost is server-side state, or a log the server can replay from.
- Periodic re-subscription. Use a deliberately short deadline, such as a few minutes, and have the client reconnect on schedule. This bounds the lifetime of each stream and spreads load across server instances. The cost is a small gap or duplicate window at each handover unless you also send tokens.
const grpc = require('@grpc/grpc-js');
// Client: the deadline is an absolute Date, not a duration.
const stream = client.subscribe({ topic: 'orders' }, {
deadline: new Date(Date.now() + 30_000),
});
stream.on('data', (msg) => handle(msg));
stream.on('error', (err) => {
if (err.code === grpc.status.DEADLINE_EXCEEDED) {
scheduleRetryWithBackoff(); // we gave up waiting; not a server fault
} else {
reportFailure(err);
}
});
// Server: budget work and propagate the remaining time downstream.
const SAFETY_MARGIN_MS = 100; // network + serialization headroom
function getOrderStream(call) {
// getDeadline() may return a Date or a number; Infinity means no deadline.
const d = call.getDeadline();
const deadlineMs = d instanceof Date ? d.getTime() : d;
const remaining = deadlineMs - Date.now();
if (remaining < 500) { // too little time to start an expensive lookup
call.destroy({ code: grpc.status.DEADLINE_EXCEEDED, details: 'insufficient budget' });
return;
}
// Downstream stops when our caller gives up, minus a margin.
const downstreamDeadline = Number.isFinite(deadlineMs)
? new Date(deadlineMs - SAFETY_MARGIN_MS)
: new Date(Date.now() + 10_000); // caller set none; cap it ourselves
inventoryClient.lookup({ id: call.request.id }, { deadline: downstreamDeadline }, (err, res) => {
if (err) return call.destroy(err);
call.write(res);
call.end();
});
}
A client sets an absolute deadline and treats DEADLINE_EXCEEDED as a retryable condition, while the server reads call.getDeadline() to check its remaining budget and gives a downstream call that budget minus a safety margin.
How should a grpc-js handler clean up on cancellation, deadline expiry and errors?
Every streaming handler needs one exit path. Write a single cleanup() function that stops the producer loop, closes any database cursor or broker subscription, and clears timers such as heartbeats. Guard it with a boolean so a second call does nothing. Several triggers can fire close together, for example a deadline expiring while a cursor read is failing, and closing a cursor twice often throws.
Attach cleanup() to the call's 'cancelled' event. In grpc-js, an explicit client cancel, an expired deadline and a dropped connection all end up there. Also check call.cancelled before each write() and after every await. Without this check the producer keeps running. The transport enforces the deadline by ending the RPC and sending DEADLINE_EXCEEDED to the client, but it does not stop your code. A handler that never checks for cancellation keeps querying the database and building messages for a stream that no longer exists.
Terminate the call deliberately:
- On success, call
end()after the last message. - On failure, emit an error carrying a status object with a
codeand human-readabledetails, then run cleanup. - Write nothing after
end(), after an error, or after cancellation. - Catch every rejection inside the async producer. An exception thrown after the call has finished has no stream to report to. It becomes an unhandled rejection that can crash the process.
A failed stream has usually delivered part of its data already. When the client retries, it receives those messages again. Give each message a sequence number or a stable ID so the client can resume from the last one it processed, or make consumers idempotent so that applying a message twice has no extra effect.
import { ServerWritableStream, status } from '@grpc/grpc-js';
import { once } from 'node:events';
type Row = { id: string; payload: string };
declare function openCursor(topic: string): { next(): Promise<Row | null>; close(): Promise<void> };
export function streamEvents(call: ServerWritableStream<{ topic: string }, unknown>): void {
const cursor = openCursor(call.request.topic);
const heartbeat = setInterval(() => { /* emit keepalive metric */ }, 15_000);
const abort = new AbortController(); // unblocks a pending 'drain' wait
let cleaned = false;
// One idempotent exit path: safe to call from cancel, error and success.
const cleanup = () => {
if (cleaned) return;
cleaned = true;
abort.abort();
clearInterval(heartbeat);
cursor.close().catch(() => {}); // a failed or closed cursor must not throw here
};
// Client cancel, deadline expiry and connection loss all emit 'cancelled'.
call.on('cancelled', cleanup);
(async () => {
let seq = 0;
for (let row = await cursor.next(); row; row = await cursor.next()) {
if (call.cancelled) return; // re-check after every await
// seq lets the client resume after a retry instead of re-applying messages
if (!call.write({ seq: ++seq, id: row.id, payload: row.payload })) {
await once(call, 'drain', { signal: abort.signal }); // respect backpressure
if (call.cancelled) return;
}
}
call.end(); // success: end only after the last message
})()
.catch((err: Error) => {
// Every rejection is caught; report it only if the call is still open.
if (!call.cancelled) call.emit('error', { code: status.UNAVAILABLE, details: `stream failed: ${err.message}` });
})
.finally(cleanup);
}
A grpc-js server-streaming handler that routes cancellation, deadline expiry and failures to one guarded cleanup function, checks call.cancelled after every await, ends with end() or a status error, and tags each message with a sequence number for safe client resume.
How do you keep long-lived gRPC streams healthy?
Even with resumable messages, a stream that dies silently is a problem, and long-lived streams are the ones most likely to die that way. Load balancers, reverse proxies and NAT gateways track connections and drop the ones that look idle after a timeout. A server stream that only sends when something changes can go quiet for minutes, and the intermediary may then discard its state without telling either side. The client keeps waiting on a connection that no longer exists, and nothing errors until the next write fails.
Keepalive pings
HTTP/2 PING frames detect this. In grpc-js you enable them with channel options: grpc.keepalive_time_ms sets how long the connection can be idle before a ping is sent, and grpc.keepalive_timeout_ms sets how long to wait for the acknowledgement before the transport is treated as dead and active calls fail with UNAVAILABLE. grpc.keepalive_permit_without_calls allows pings when no call is active.
Do not set the interval aggressively. Servers and intermediaries often enforce a minimum ping interval. A client that pings faster than that minimum gets a GOAWAY, typically with a "too many pings" reason, and the connection closes. That is the outage keepalive was meant to prevent. Find the minimum the server and every proxy on the path allow. Then set the client interval comfortably above it and below the shortest idle timeout.
Plan for reconnection
Treat every stream as temporary. When a stream ends with UNAVAILABLE or a deadline error, reconnect with exponential backoff and jitter. Pass the last processed sequence number or cursor in the new request so the server resumes from that point instead of replaying everything or dropping messages. Some deployments also end streams on purpose at intervals, for example to rebalance load or during a server drain. The client should handle those closures the same way.
What are the alternatives to gRPC streaming, and what do they cost to build and run?
Before choosing server streaming, compare it with the options below. Several of them avoid the hardest problems covered in this article.
| Option | Best fit | Flow control | Browser support | Operational complexity |
|---|---|---|---|---|
| Paginated unary calls | Bounded result sets, batch exports | Client pulls one page at a time | Through a gRPC-Web proxy | Low |
| Server streaming | Large or ongoing server-to-client feeds | HTTP/2 windows plus write-side checks | Through a gRPC-Web proxy | Medium |
| Bidirectional streaming | Interactive sessions, app-level acks | HTTP/2 in both directions | Not supported over gRPC-Web | High |
| SSE / WebSockets | Browser-facing push | TCP only; you add acks yourself | Native | Medium |
| Message broker, consumed directly | Durable fan-out, replay | Consumer-paced | No | High (more infrastructure) |
Streaming costs more to build than pagination. You have to write backpressure handling, cleanup paths, resume tokens and failure tests. That work is worth doing because without it, stream failures are hard to trace. An unhandled slow consumer shows up later as a memory incident. A missed cancel listener leaves a database cursor open. A stream that ends with no status forces someone to read packet captures. If you test these paths before release, each one becomes a known, logged outcome instead of an on-call investigation.
Failure-path test plan
- Slow consumer: Pause reads on the client. Assert that the server stops producing once
write()returns false. - Mid-stream client cancel: Cancel after N messages. Assert that the
cancelledhandler ran and that the cursor is closed. - Deadline expiry during a slow producer step: Set a short deadline and stall the producer. Assert that the client gets
DEADLINE_EXCEEDEDand that the server-side work stops. - Server error after a partial response: Throw after some messages have been sent. Assert that the client receives the messages already written, followed by the error status.
For every test, run it in a loop. Assert that heap usage levels off and stops growing, and that the count of open handles returns to its baseline.
How can gRPC server streaming save engineering time?
The time savings come from work you no longer have to write or maintain for each endpoint. They do not come from the transport being faster.
- No hand-rolled pagination or polling. With unary pagination, every endpoint needs a page-token format, a stateless resume query and a client loop that requests the next page. A polling feed also needs an interval to tune and a way to dedupe overlapping results. A stream replaces this with one call and one consumer loop. You still need a resume token, but only for reconnects, not for every page.
- Flow control is already built. HTTP/2 windows and Node.js stream backpressure already exist. Your job is to honour
write()'s return value or usepipeline(). With SSE or WebSockets, you design and test an application-level ack or credit scheme yourself. - One contract, generated clients. The
.protofile defines the request, the message type and the streaming shape. Clients in other languages are generated from the same file, so you do not write and keep in sync a separate client per consumer. - Cancellation and deadlines are part of the protocol. Deadlines travel with the call, and a cancelled client is visible to the server through the
'cancelled'event andcall.cancelled. You do not invent timeout headers or "client went away" detection. You still have to wire them to cleanup, as shown earlier. - Fewer vague incidents. The patterns in this article each map to a test in the plan above: bounded buffers, one cleanup path, explicit status codes and sequence numbers. Failures then show up as specific status codes and log lines that someone can act on, instead of slow memory growth or leaked cursors.
These savings depend on the use case. For small, bounded responses, a unary call saves more time than a stream, because the streaming-specific work shown here becomes overhead you did not need.
Illustrative estimate, with assumed inputs only: assume a team runs 5 feed-style endpoints. Assume each one, built as paginated or polling unary calls, needs about 2 days for its page-token logic, client loop and dedupe tests. Assume a shared streaming pattern built once takes about 4 days for the handler template, cleanup helper, resume tokens and failure tests, plus about 0.5 days for each endpoint that adopts it. That gives about 10 days for the per-endpoint approach against about 6.5 days for streaming. The gap grows with each additional endpoint. It shrinks or reverses if the team has only one or two such endpoints or already has a pagination library. Replace these assumptions with your own figures before you rely on the comparison.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to grpc streaming nodejs, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a route's URL or method automatically notifies the services that consume it — no hunting through repos for callers.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Consumers learn about a breaking change at review time rather than from production errors.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.