Server-Sent Events vs WebSockets: Proxies, Load Balancing and Scaling

How SSE and WebSockets work on the wire

An SSE stream is an ordinary HTTP response that never finishes. The server replies 200 with Content-Type: text/event-stream and then writes UTF-8 text events. Each event is a set of field: value lines followed by a blank line. A minimal event is id: 42, event: price, data: {"sym":"ABC"} and a terminating empty line. A retry: 5000 line sets the client's reconnect delay in milliseconds.

A WebSocket starts as an HTTP request: GET /ws with Upgrade: websocket, Connection: Upgrade, Sec-WebSocket-Key and Sec-WebSocket-Version: 13. The server answers 101 Switching Protocols with a matching Sec-WebSocket-Accept. After that, the connection stops carrying HTTP. Both sides exchange binary frames with opcodes, client-side masking, ping/pong and close codes.

This difference decides how intermediaries treat each one. SSE stays inside HTTP semantics, so proxies, access logs, auth middleware and tracing see a normal request. WebSockets need every hop to pass the Upgrade header and then forward opaque bytes, which HTTP-level tooling can't inspect.

EventSource reconnects on its own, waits the retry interval, and sends the last received id as Last-Event-ID. WebSocket clients get only a close event. You have to write the reconnect, backoff, jitter and resume-from-offset logic yourself.

Copy# --- SSE: one HTTP response that stays open ---
HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache

retry: 5000

id: 42
event: price
data: {"sym":"ABC","px":101.25}

id: 43
event: price
data: {"sym":"ABC","px":101.30}

# (response body continues; a blank line ends each event)

# --- SSE reconnect: EventSource resends the last id ---
GET /prices HTTP/1.1
Host: api.example.com
Accept: text/event-stream
Last-Event-ID: 43

# --- WebSocket: HTTP handshake request ---
GET /ws HTTP/1.1
Host: api.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13

# --- WebSocket: handshake response ---
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

# After the 101, the connection carries WebSocket frames, not HTTP.

The bytes on the wire for an SSE stream with retry and id fields, the Last-Event-ID header a reconnecting EventSource sends, and a WebSocket Upgrade handshake that uses the sample key and accept values from RFC 6455.

Getting push through proxies, CDNs and load balancers

Each hop between client and server can break a push connection in its own way.

Buffering. Reverse proxies often buffer upstream responses. For SSE, events then sit in the proxy until a buffer fills, so the client sees nothing. Turn buffering off on streaming routes only. In nginx, set proxy_buffering off on the route, or have the app send X-Accel-Buffering: no. Compression middleware can hold output too, so exclude text/event-stream from it.

Idle timeouts. CDNs, load balancers, ingress controllers and app servers each close connections that stay quiet for too long, and the shortest timeout wins. nginx's proxy_read_timeout defaults to 60 seconds. Send an SSE comment line (: ping) or a WebSocket ping well inside the smallest timeout. Then align all the timeouts so you know which hop will close a connection first.

Upgrade handling. Upgrade and Connection are hop-by-hop headers, so a proxy does not forward them unless you configure it to. For WebSocket routes, set proxy_http_version 1.1 and pass both headers explicitly.

Connection limits. Over HTTP/1.1, browsers allow about six connections per origin. Each SSE stream holds one, so a few open tabs can starve the page's other requests. HTTP/2 multiplexes many streams over one connection, which removes most of this limit. WebSockets usually still run over HTTP/1.1, because support for bootstrapping them over HTTP/2 is uneven across browsers, proxies and servers.

Hostile middleboxes. Corporate proxies and antivirus products can buffer SSE or drop WebSocket upgrades, and you cannot reconfigure them. A long-polling fallback works around them, but it is a second transport. You have to write it, test it and debug it, with its own ordering and duplicate-delivery cases.

The configuration example below shows an SSE route and a WebSocket route side by side.

Copy# Inside the http {} block. Set Connection based on whether the client asked to upgrade.
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

upstream app_backend {
    server 10.0.0.12:8080;
}

server {
    listen 80;
    server_name push.example.internal;

    # SSE route: stream events as soon as the app writes them
    location /events {
        proxy_pass
        proxy_http_version 1.1;
        proxy_set_header Connection "";   # keep the upstream connection open
        proxy_buffering off;              # do not hold events in proxy buffers
        proxy_cache off;
        gzip off;                         # compression can hold output too
        proxy_read_timeout 300s;          # longer than the app's heartbeat interval
    }

    # WebSocket route: forward hop-by-hop upgrade headers explicitly
    location /ws {
        proxy_pass
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header Host $host;
        proxy_read_timeout 300s;          # align with LB/CDN idle timeouts; ping inside it
    }
}

An nginx configuration that disables buffering and compression on an SSE route, forwards the Upgrade and Connection headers on a WebSocket route, and raises the read timeout on both routes so the app's heartbeats keep quiet connections open.

Load balancing and fan-out for long-lived connections

A load balancer picks a backend once, at connect time, and that choice holds for the life of the connection. Round-robin therefore balances connection attempts, not open connections. After you scale from four nodes to six, the two new nodes receive only new connects while the old ones keep their full load. A deploy or a crashed node pushes its clients onto whichever survivors were up at that moment. The imbalance persists until clients reconnect for some other reason, so least-connections balancing helps, but only for new arrivals.

Correcting this takes deliberate server behaviour:

• Draining: stop accepting new connections on a node marked for removal, then close existing ones gradually rather than all at once.
• Server-initiated reconnects with jitter: periodically close long-lived connections after a randomized lifetime so load redistributes. Spread the timing so thousands of clients don't reconnect in the same second.
• Per-instance caps: reject or redirect above a fixed connection count so one node can't absorb a reconnect storm.

All of this works only if any node can serve any client. To get there, separate connection-holding gateways from event producers. Producers publish to a pub/sub layer, and gateways subscribe and forward. Sticky sessions then become optional for both SSE and WebSockets.

The extra hop costs latency. Ordering is typically only guaranteed per partition or topic. Fire-and-forget pub/sub drops events while a gateway is reconnecting. SSE resumption via Last-Event-ID therefore needs a replay buffer or durable log keyed by event ID that any gateway can read.

Copy Clients (SSE / WebSocket)
 +--------+ +--------+ +--------+ +--------+
 | client | | client | | client | | client |
 +---+----+ +---+----+ +---+----+ +---+----+
     v          v          v          v  reconnect w/ jitter + Last-Event-ID
 +--------------------------------------------------+
 |  Load balancer (least-conn; stickiness optional) |
 +-------+------------------+------------------+----+
         v                  v                  v
 +-------+-------+  +-------+-------+  +-------+-------+
 | Gateway 1     |  | Gateway 2     |  | Gateway 3 NEW |
 | cap: N conns  |  | DRAINING      |  | cap: N conns  |
 | full load     |  | no new conns  |  | fills only on |
 | holds conns   |  | close gradual |  | reconnect     |
 +-------+-------+  +-------+-------+  +-------+-------+
         ^ subscribe        ^                  ^  replay by event ID
 +-------+------------------+------------------+---------+
 | Pub/sub layer (ordering per partition/topic)           |
 | + replay buffer / durable log keyed by event ID        |
 | gateways resume from Last-Event-ID                     |
 +--------------------------+-----------------------------+
                            ^ publish
         +------------------+------------------+
         |                  |                  |
 +-------+-------+  +-------+-------+  +-------+-------+
 | Producer A    |  | Producer B    |  | Producer C    |
 +---------------+  +---------------+  +---------------+

Connection-holding gateways sit behind the load balancer and subscribe to a pub/sub layer fed by producers, so any gateway can serve any client, drain or accept jittered reconnects, and resume SSE streams from a replay log keyed by event ID.

Implementing resumable SSE with heartbeats and a replay buffer

Server side

Give every event a monotonic ID from the log or partition offset, not a per-connection counter, so any gateway can resume any client. Keep a bounded replay buffer (a ring buffer per topic, or a read from the durable log) sized to your expected reconnect window. When a request arrives with a Last-Event-ID header, replay everything after that ID before switching to live events. If the ID has already been evicted, send a named event such as resync so the client refetches full state. Silently skipping the gap would leave the client with missing data and no indication of it.

Send retry: once at stream start to set the browser's reconnect delay in milliseconds. Emit a comment line (: ping) every 15–30 seconds. It keeps idle-timeout proxies from cutting the connection, and a failed write is how you detect dead clients. On write error or request cancellation, unsubscribe immediately so the fan-out layer stops buffering for that connection.

Client-to-server and ordering

Commands go over plain HTTP POST. The POST response and the stream event it causes travel on different connections, so their arrival order is not guaranteed. Have the POST return the resulting event ID. The client then treats its local state as confirmed only once the stream has reached that ID.

WebSocket contrast

A WebSocket client gets none of this for free. It must:

• implement reconnection with exponential backoff and jitter
• persist the last sequence number it processed
• send a resume message after reconnecting
• run application-level pings, because browsers don't expose protocol ping frames

Copyconst http = require('node:http');

const MAX_BUFFER = 1000; // size to the expected reconnect window
const buffer = [];       // ring of {id, data}; in production read from the durable log
let nextId = 1;          // stand-in for a log/partition offset, not a per-connection counter
const clients = new Set();

const frame = (ev) => `id: ${ev.id}\ndata: ${ev.data}\n\n`;

function publish(payload) {
  const ev = { id: nextId++, data: JSON.stringify(payload) };
  buffer.push(ev);
  if (buffer.length > MAX_BUFFER) buffer.shift();
  for (const res of clients) res.write(frame(ev));
  return ev.id;
}

http.createServer((req, res) => {
  if (req.method === 'POST' && req.url === '/commands') {
    let body = '';
    req.on('data', (c) => (body += c));
    // Return the event ID so the client can wait until the stream reaches it
    req.on('end', () => res.end(JSON.stringify({ eventId: publish(JSON.parse(body)) })));
    return;
  }
  if (req.url !== '/events') return res.writeHead(404).end();

  res.writeHead(200, { 'Content-Type': 'text/event-stream', 'Cache-Control': 'no-cache' });
  res.write('retry: 3000\n\n'); // browser reconnect delay in ms

  const last = Number(req.headers['last-event-id']);
  if (Number.isInteger(last) && last > 0) {
    const oldest = buffer.length ? buffer[0].id : nextId;
    if (last < oldest - 1) res.write('event: resync\ndata: {}\n\n'); // gap evicted: tell client to refetch
    else for (const ev of buffer) if (ev.id > last) res.write(frame(ev));
  }
  clients.add(res); // replay and join happen in one tick, so no live event slips between them

  const ping = setInterval(() => res.write(': ping\n\n'), 20000); // keeps idle proxies open
  res.on('close', () => { clearInterval(ping); clients.delete(res); }); // stop fan-out immediately
}).listen(8080);

A dependency-free Node.js SSE server that assigns monotonic IDs, replays from a bounded buffer on Last-Event-ID (or emits resync when the gap is evicted), sets retry, sends heartbeat comments, cleans up on disconnect, and returns the resulting event ID from a command POST.

Capacity planning, auth and observability

Each open connection holds a socket, a file descriptor, kernel buffers, TLS state and whatever your framework allocates per stream. Raise the process and system file descriptor limits above your peak connection count, or new connections fail.

Illustrative estimate. Assume 100,000 connections, 30 KB of memory per connection, and a heartbeat every 30 seconds at roughly 100 bytes on the wire after framing and TLS overhead. That needs about 3 GB of RAM and 100,000+ descriptors. Heartbeats alone produce about 3,300 writes per second and about 330 KB/s of egress.

Set the heartbeat interval below the shortest idle timeout on the path, whether that is a proxy, load balancer or carrier NAT. Half of that timeout is a reasonable margin. Shorter intervals keep connections alive more reliably, but each heartbeat costs something:

• On mobile devices, every heartbeat wakes the radio and holds it in a high-power state, which drains the battery.
• On the server, every heartbeat is a timer wakeup and a syscall for every connection.

With WebSockets, you have to rebuild those controls at the message level. That means per-message logging, rate limits, payload validation and latency metrics keyed by message type.

When to use SSE, when to use WebSockets, and when to avoid both

The deciding factor is usually traffic direction. If the server talks and the client mostly listens, SSE is plain HTTP that your existing controls already understand. If the client sends often or needs low-latency round trips, the per-request overhead of SSE plus separate POSTs adds up, and a WebSocket is the better fit.

SSE has hard limits. It is text-only, so binary data must be encoded, typically as base64, which inflates size by about a third and costs CPU. Client-to-server messages need separate HTTP requests, which bring their own auth, routing and ordering concerns.

When to avoid both

Serverless and request-duration-limited platforms cap response lifetimes, so long-lived streams get cut off and reconnect storms follow. Hostile intermediaries, such as corporate proxies that buffer responses or kill idle connections, break both protocols. In those cases, consider these alternatives:

• Short polling: simple and cacheable. It suits data that changes slowly.
• Long polling: each request is held open until an event arrives. It survives most proxies.
• A managed push service: offloads connection state and fan-out to a vendor.
• HTTP streaming with fetch: reads the response body as a stream. Unlike EventSource, it allows POST and custom headers.

Impact on development cost and engineering time

SSE's main cost advantage is that it is ordinary HTTP. The auth middleware, access logs, rate limiters, tracing and WAF rules you already run apply to the stream request without changes. Reconnection is built into EventSource. If you use the Last-Event-ID replay from section 4, resuming after a drop takes only a server-side lookup and no client protocol work.

WebSockets move most of that work into your own code:

• Reconnect logic: you build backoff, jitter and resume semantics yourself.
• Observability: an access log records one long-lived request, so you need per-message logging and metrics to see what actually happened.
• Security controls: you need explicit Origin checks, per-message authorization, message size limits and token re-validation on long sessions.
• Infrastructure: every hop must be upgrade-aware and configured for long idle timeouts.

Both protocols share the costs that come from long-lived connections:

• Slow deploys: you can't finish a deploy until connections drain.
• Weak autoscaling signals: CPU tracks connection count poorly, and new instances receive no traffic until clients reconnect.
• A pub/sub dependency: fan-out across instances requires a broker that you must operate and monitor.

Illustrative estimate (assumed numbers, not a benchmark): rollout time ≈ ceil(instances ÷ instances replaced per batch) × drain timeout. With 20 instances, 5 per batch and a 120 s drain: ceil(20 ÷ 5) × 120 s = 4 × 120 s = 480 s, or 8 minutes per deploy before any startup time.

Recommendation: default to SSE for server-to-client push and plain HTTP POST for client-to-server actions. Adopt WebSockets only when you need low-latency, high-frequency bidirectional messaging, such as collaborative editing or multiplayer state, and that need justifies the extra operational cost.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to server-sent events vs websockets, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.