Node.js Stream Large JSON Response Without Memory Spikes

Why res.json(bigArray) causes memory spikes in APIs

Calling res.json(rows) looks like one step, but it forces the whole response to exist in memory several times at once. First, the driver or ORM has already turned every row into a JavaScript object. Next, JSON.stringify walks that object graph and builds one complete string. Then the framework converts the string into a Buffer so it can compute Content-Length and write bytes to the socket. The original array is still referenced by your handler, so the garbage collector cannot free it until the response has been handed off. At the peak, all three copies are alive together.

Illustrative estimate (assumed inputs, not a measurement)

Assume 200,000 rows that each serialize to about 1 KB of ASCII JSON, for a payload of roughly 200 MB.

• Object graph: JS objects carry per-object and per-property overhead. Assume they take about 3× their JSON size, which is roughly 600 MB.
• Serialized string: about 200 MB. Non-Latin-1 text can double this, because the engine then stores two bytes per character.
• Buffer copy: about 200 MB.

The peak is roughly 1 GB, about 5× the payload, for a single request. Ten concurrent exports would multiply that. Very large payloads can also exceed the engine's maximum string length and fail outright.

What streaming means here

Streaming a large JSON response means producing and sending the body in small chunks as rows arrive. Memory then stays bounded by the chunk size and socket buffers rather than by the result set. Backend and platform engineers need this when they run exports, reports, or list endpoints whose result sets have no fixed upper bound.

Copyconst http = require('node:http');
const { Readable } = require('node:stream');
const { pipeline } = require('node:stream/promises');

// Placeholder data source: yields rows one at a time, like a DB cursor would.
async function* fetchRows(count) {
  for (let i = 0; i < count; i++) {
    yield { id: i, name: `item-${i}`, createdAt: '2026-01-01T00:00:00Z' };
  }
}

// Buffered: the array, the full JSON string and the Buffer are all alive at the peak.
async function buffered(res) {
  const rows = [];
  for await (const row of fetchRows(200000)) rows.push(row);
  const body = Buffer.from(JSON.stringify(rows));
  res.writeHead(200, { 'Content-Type': 'application/json', 'Content-Length': body.length });
  res.end(body);
}

// Streamed: serialize one row at a time and emit valid JSON array syntax around them.
async function* toJsonArray(rows) {
  yield '[';
  let first = true;
  for await (const row of rows) {
    yield (first ? '' : ',') + JSON.stringify(row);
    first = false;
  }
  yield ']';
}

async function streamed(res) {
  // No Content-Length: Node falls back to chunked transfer encoding.
  res.writeHead(200, { 'Content-Type': 'application/json' });
  // pipeline pauses the generator when the socket buffer is full (backpressure).
  await pipeline(Readable.from(toJsonArray(fetchRows(200000))), res);
}

http.createServer((req, res) => {
  const handler = req.url === '/export/buffered' ? buffered : streamed;
  handler(res).catch((err) => { console.error(err); res.destroy(err); });
}).listen(3000);

The same 200,000-row export is served two ways: the buffered handler holds the array, the full string and a Buffer at the same time, while the streamed handler serializes one row at a time and uses backpressure, so memory stays bounded.

How an end-to-end streaming response works

A streaming response is a chain of four stages: an incremental source (a database cursor or any async iterator that yields rows), a transform that serializes each row to bytes, the HTTP response, and the client's TCP socket. Data flows downstream, and the signal to slow down flows back upstream.

That signal starts at res.write(). Its boolean return value does not mean "the write succeeded." false means the response's internal buffer has reached its highWaterMark. The chunk is still queued and will be sent, but you should stop writing until the stream emits 'drain'. If code ignores false and keeps writing, every extra chunk accumulates in process memory. That is the same spike as res.json(bigArray), only spread over time.

Wiring write()/'drain' by hand is error-prone. Readable.from(asyncIterable) turns an async iterator into a readable stream that calls the iterator's next() only when downstream asks for more data. stream.pipeline(source, transform, res) connects the stages and respects each stage's buffer limits. If any stage fails or the client disconnects, it destroys every stream in the chain. That lets the iterator's cleanup run, so a cursor can be closed.

Here is how a slow client on a weak mobile connection plays out:

1. The client acknowledges data slowly, so the kernel's send buffer fills.
2. The response's buffer passes its high-water mark, so the pipeline stops feeding the transform.
3. The transform's buffer fills, so Readable.from stops calling next().
4. The cursor stops fetching batches.

The database read now runs at the client's pace. Memory stays capped at the sum of a few small buffers.

Copy  data flows down (v)                  backpressure flows up (^)          ^
         v                                                                |
 +----------------------------------+                                     |
 | DB cursor / async iterator       |  4. cursor stops fetching batches   |
 +----------------------------------+                                     |
         | rows from the cursor                                           |
         v                                                                |
 +----------------------------------+                                     |
 | Readable.from(asyncIterable)     |  3. next() is not called while      |
 |   calls next() only on demand    |     downstream buffers are full     |
 +----------------------------------+                                     |
         | row objects                                                    |
         v                                                                |
 +----------------------------------+                                     |
 | Transform: serialize each row    |  2. buffer fills; pipeline stops    |
 |   internal buffer (hwm)          |     feeding the transform           |
 +----------------------------------+                                     |
         | serialized bytes                                               |
         v                                                                |
 +----------------------------------+                                     |
 | HTTP response (res)              |  res.write() returns false at hwm   |
 |   internal buffer (hwm)          |  'drain' fires -> writing resumes   |
 +----------------------------------+                                     |
         | bytes                                                          |
         v                                                                |
 +----------------------------------+                                     |
 | kernel send buffer / TCP socket  |  1. slow ACKs fill send buffer      |
 +----------------------------------+                                     |
         v                                                                |
   [ slow mobile client ] ------------------------------------------------+

Rows flow down from the cursor through Readable.from, the serializing transform and res to the client socket, while full buffers propagate backpressure upstream (steps 1-4) until the cursor stops fetching.

Sourcing rows incrementally instead of loading them all

That chain only holds if the source can actually pause. An ORM findAll() or an unbounded SELECT cannot. The driver materializes every row, the ORM builds an object per row, and only then does your code see the result. By the time the first byte reaches the transform, the peak has already happened. Streaming the serialization of an array that is already fully in memory saves almost nothing.

There are two ways to make the source pull-based.

Server-side cursors. Many drivers can expose a query as a cursor or async iterator that fetches a fixed number of rows per round trip. An async generator wraps it, reads a batch, yields each row, and requests the next batch only when the consumer calls next() again.

Keyset pagination. The generator runs a bounded query ordered by an indexed, unique key, such as WHERE id > $last ORDER BY id LIMIT 1000. It yields those rows, records the last id, and repeats until a query returns nothing. Avoid OFFSET, because the database still scans every skipped row, so each later page costs more than the one before it.

The two approaches differ in what a slow client costs you. A cursor needs one connection, usually inside a transaction, for the entire download. A client on a poor link can hold that pool slot for minutes, and the open transaction can pin a snapshot or hold locks that block cleanup and writers. Backpressure has moved the cost from memory to connection lifetime. Keyset pagination borrows a connection only for each batch and returns it between batches, at the price of losing a single consistent snapshot across the whole response.

Copyimport { Pool } from 'pg';
import { Readable } from 'node:stream';
import { pipeline } from 'node:stream/promises';

const pool = new Pool({ max: 10 }); // connection settings come from PG* env vars

// Keyset pagination: each batch borrows a pooled connection for one query only,
// so a slow client never pins a connection between batches.
async function* streamOrders(batchSize = 1000) {
  let lastId = 0;
  while (true) {
    const { rows } = await pool.query(
      'SELECT id, customer_id, total_cents, created_at FROM orders WHERE id > $1 ORDER BY id LIMIT $2',
      [lastId, batchSize]
    );
    if (rows.length === 0) return;
    // Execution pauses at each yield until the consumer pulls the next row.
    for (const row of rows) yield row;
    lastId = rows[rows.length - 1].id; // bigint ids arrive as strings; passing them back as $1 is fine
  }
}

// Serialize one row at a time into a JSON array.
async function* toJsonArray(rows) {
  let first = true;
  yield '[';
  for await (const row of rows) {
    yield (first ? '' : ',') + JSON.stringify(row);
    first = false;
  }
  yield ']';
}

export async function handler(req, res) {
  res.setHeader('Content-Type', 'application/json');
  // pipeline propagates backpressure from the socket back to the generator,
  // so the next query runs only after the client has drained the previous batch.
  await pipeline(Readable.from(toJsonArray(streamOrders())), res);
}

An async generator built on node-postgres uses keyset pagination to fetch orders in bounded batches, so pool connections are held only per query, and it feeds a backpressure-aware JSON serializer through stream.pipeline.

Implementing NDJSON streaming and framed JSON arrays with pipeline

Whichever source you chose, the response side has the same shape: a readable of rows, a Transform that serializes them, and res as the sink, connected with pipeline from stream/promises.

NDJSON

Set Content-Type: application/x-ndjson and emit JSON.stringify(row) + '\n' per record. JSON.stringify escapes newlines inside strings, so a literal newline can only end a record. A client appends incoming bytes to a buffer, parses each complete line, and keeps the incomplete tail for the next chunk. If the connection drops, the client loses at most one partial record, and every earlier record has already been parsed.

Framed JSON array

When clients expect one document, push [ before the first row and , before every later row, tracking the first row with a boolean. In flush, push ], or [] if no row arrived, so an empty result is still valid JSON rather than an empty body.

Why pipeline

If any stage errors or the client disconnects, pipeline destroys every stream in the chain and rejects its promise. Destroying the source runs its _destroy, which is where the cursor closes and the connection returns to the pool. With chained .pipe(), an early res close only unpipes the source. The source stays paused, and its cursor and connection stay open until something times them out.

Pushing one string per row costs a function call and a chunk per row. Joining 100–1,000 serialized rows per push cuts CPU and syscall overhead but raises peak memory per request, so tune the batch size against your row size and concurrency.

Copyimport { Transform } from 'node:stream';
import { pipeline } from 'node:stream/promises';
import { getRowStream } from './db.js'; // placeholder: object-mode Readable whose _destroy closes the cursor

// NDJSON: one serialized row per line; JSON.stringify escapes embedded newlines
const toNdjson = () => new Transform({
  writableObjectMode: true, // accepts row objects, emits strings
  transform(row, _enc, cb) { cb(null, JSON.stringify(row) + '\n'); },
});

// Framed array: '[' before the first row, ',' before every later row
const toJsonArray = () => {
  let first = true;
  return new Transform({
    writableObjectMode: true,
    transform(row, _enc, cb) {
      cb(null, (first ? '[' : ',') + JSON.stringify(row));
      first = false;
    },
    // No rows seen: emit '[]' so the body is still valid JSON
    flush(cb) { cb(null, first ? '[]' : ']'); },
  });
};

export async function handler(req, res) {
  const ndjson = (req.headers.accept ?? '').includes('application/x-ndjson');
  res.setHeader('Content-Type', ndjson ? 'application/x-ndjson' : 'application/json');

  try {
    // On error or client disconnect, pipeline destroys all three streams,
    // so the source's _destroy runs and the cursor/connection is released.
    await pipeline(getRowStream(), ndjson ? toNdjson() : toJsonArray(), res);
  } catch (err) {
    // A client hanging up mid-stream is expected; log anything else
    if (err.code !== 'ERR_STREAM_PREMATURE_CLOSE') console.error('stream failed', err);
  }
}

An HTTP handler that streams rows as NDJSON or as a framed JSON array (emitting [] for an empty result), connected with pipeline from stream/promises so every stage is destroyed on error or client disconnect.

Handling errors after headers are sent and the proxy and compression pitfalls

Once the first byte is flushed, the status line and headers have been sent. If the database cursor fails on row 40,000, the client has already received a 200, and the server cannot change it to a 500. In practice you have two options.

Abort the connection. Calling res.destroy(err), or letting pipeline destroy the response on failure, resets the socket or ends a chunked response without its terminating chunk. HTTP clients usually report this as a premature close, which keeps a partial body from looking successful.

Emit a trailing error record. With NDJSON you can write {"type":"error","message":"..."} as the last line and then end the response cleanly. This produces a readable diagnosis, but only clients that check for this record will notice the failure.

Whichever you choose, document how clients should detect truncation. For framed arrays, a missing closing ] makes the parse fail. For NDJSON, finish every successful stream with a summary record such as {"type":"end","count":N} and tell consumers to treat output without it as incomplete.

Layers that quietly buffer

Your handler can stream correctly while something downstream collects the whole body. Gzip middleware compresses in blocks and holds output until its buffer fills or it is explicitly flushed, so small rows can sit in memory for a long time. A reverse proxy with response buffering enabled reads the full upstream body before sending anything to the client. Framework helpers that serialize or set Content-Length must materialize the payload first. Check the time to first byte as measured at the client, not at the handler.

Verifying the fix under load, slow clients and disconnects

Instrumentation comes first. Log process.memoryUsage() on a fixed interval (every second is enough) while a large export runs, and record heapUsed and rss side by side. heapUsed shows JavaScript objects your code is holding. rss also includes Buffers and native allocations, which is where socket write queues live. A streaming handler should hold both lines roughly flat for the whole export, with small sawtooth dips as garbage collection runs. If heapUsed climbs in proportion to rows sent, something is accumulating them, usually an array or a string concatenation. If heapUsed is flat but rss keeps rising, Buffers are piling up, which usually means a stream is ignoring backpressure.

Next, test a slow client. Run curl --limit-rate 50k against the endpoint so the socket drains far slower than the database produces rows. With backpressure working, write() returns false, pipeline pauses the source, and memory stays flat even though the export now takes minutes. A growing curve under throttling, but not at full speed, points to a hand-written loop that keeps writing without awaiting drain.

Then test a disconnect. Interrupt curl with Ctrl-C partway through the export. Confirm that the handler logs the close, that the source stream is destroyed, and that the database no longer shows the query or cursor as active. A leaked cursor here eventually exhausts the connection pool.

Run these checks in CI against a fixed dataset. A new middleware that buffers the response body turns the flat curve into a ramp, and the test fails before the change reaches production.

When to avoid streaming, alternatives, and impact on cost and engineering time

Streaming is the wrong tool when the client needs the whole document before it can act, or when the response must be cached. NDJSON is not valid JSON as a whole, so any client that calls JSON.parse on the body breaks. Switching formats is therefore an API contract change, and it needs a new version or content negotiation on the Accept header. A streamed body has no Content-Length and usually no ETag. Caches skip it, a dropped connection restarts from row zero, and a retry replays the entire query. Streaming also caps memory, not duration or size. A slow 30-minute response still holds a socket and a database cursor for 30 minutes.

Streaming has development costs. Stack traces from a failed transform show where pipeline reported the error, not the row that caused it. Clients need an incremental parser. Connection pools must be sized for the slowest client, because each in-flight stream holds a database connection until the last byte is sent.

The payoff is fewer out-of-memory incidents. For medium-sized payloads, you also avoid building and operating an export job pipeline.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to node.js stream large json response, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.