Skip to main content

Command Palette

Search for a command to run...

Server Sent Events(SSE): The Backbone of LLM Token Streaming

Updated
18 min readView as Markdown
Server Sent Events(SSE): The Backbone of LLM Token Streaming
💡
If you are here just to read about LLM streaming, jump to section 12

From first principles to production-grade architecture.
Covers: protocol internals, Express server setup, multi-client forwarding, debugging, CORS, buffering pitfalls, and Windows quirks.


1. What Is SSE and When Should You Use It?

Server-Sent Events (SSE) is an HTTP/1.1 feature that lets a server push a continuous stream of UTF-8 text events to a client over a single, persistent connection — without the client polling.

The browser exposes this through the EventSource API. Once you new EventSource('/endpoint'), the browser fires onmessage every time the server calls res.write(...).

Mental model

Use SSE when:

  • You need server → client push only (live feeds, notifications, progress bars, logs).

  • You want automatic reconnection for free (browser EventSource handles it).

  • You need something simpler than WebSockets — SSE rides plain HTTP, works through corporate proxies, requires no upgrade handshake, and doesn't need a special library on the client.

Do NOT use SSE when:

  • You need bidirectional real-time communication → use WebSockets.

  • You need to push binary data (images, audio) → use WebSockets or chunked HTTP.

  • You have >6 tabs open on the same domain over HTTP/1.1 (browser cap) → upgrade to HTTP/2 or use WebSockets.


2. The SSE Protocol Internals

SSE is dead simple — it's plain text over a persistent HTTP connection. You only need to know four things:

2.1 Required Response Headers

Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive

These tell the browser: "Don't buffer this, don't cache it, and keep the TCP socket open."

2.2 Event Block Format

Every event is one or more field lines, terminated by a blank line (\n\n):

data: your payload here\n\n

Multi-line payloads:

data: line one\n
data: line two\n\n

Named events (the client must use addEventListener, not onmessage):

event: update\n
data: {"status":"ok"}\n\n

Event IDs (allow the client to resume from a specific point on reconnect):

id: 42\n
data: {"msg":"checkpoint"}\n\n

Retry hint (tells the browser how long to wait before reconnecting, in ms):

retry: 3000\n
data: {"msg":"reconnect in 3s"}\n\n

2.3 Strict Whitespace Rules

Rule Correct Broken
Space after colon data: payload data:payload
Double newline to end block ...\n\n ...\n
No binary content UTF-8 strings only Buffer objects

2.4 The Last-Event-ID Header

On reconnect, the browser automatically sends:

Last-Event-ID: 42

Your server can read req.headers['last-event-id'] and resume streaming from that point — enabling lossless reconnection.


3. Building the SSE Server (Express)

3.1 Minimal Working Server

import express from "express";

const app = express();

app.get("/stream-content", (req, res) => {
  // Step 1: Set the three mandatory SSE headers
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.setHeader("Connection", "keep-alive");

  let counter = 1;

  // Step 2: Push an event every second
  const interval = setInterval(() => {
    res.write(`data: chunk object ${counter++}\n\n`);
  }, 1000);

  // Step 3: Clean up when the client disconnects
  req.on("close", () => {
    clearInterval(interval);
    res.end();
  });
});

app.listen(4000, () => console.log("SSE Server running on port 4000"));

Why req.on('close') not res.on('close')?
req represents the incoming connection. When the browser navigates away or closes the tab, Node.js emits close on the request object. Using req.on('close') is the canonical pattern — always clean up your timers and database subscriptions here to prevent memory leaks.

3.2 Sending Structured JSON Events

const interval = setInterval(() => {
  const payload = JSON.stringify({
    timestamp: new Date().toISOString(),
    value: Math.random() * 100,
  });
  res.write(`data: ${payload}\n\n`);
}, 1000);

On the client:

es.onmessage = (event) => {
  const { timestamp, value } = JSON.parse(event.data);
  console.log(timestamp, value);
};

3.3 Using Named Events

// Server
res.write(`event: priceUpdate\ndata: {"symbol":"BTC","price":67000}\n\n`);
res.write(`event: alert\ndata: {"level":"warn","msg":"High load"}\n\n`);
// Client — onmessage will NOT fire for named events
es.addEventListener("priceUpdate", (e) => {
  const { symbol, price } = JSON.parse(e.data);
});

es.addEventListener("alert", (e) => {
  const { level, msg } = JSON.parse(e.data);
});

Common trap: If your server sends event: update\n, but your client only has es.onmessage, nothing logs. onmessage only fires for unnamed (default) events.

3.4 Resumable Streams with Event IDs

app.get("/stream-content", (req, res) => {
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.setHeader("Connection", "keep-alive");

  // Check if client is resuming
  const lastId = parseInt(req.headers["last-event-id"] ?? "0", 10);
  let counter = lastId + 1;

  const interval = setInterval(() => {
    res.write(`id: \({counter}\ndata: event \){counter}\n\n`);
    counter++;
  }, 1000);

  req.on("close", () => {
    clearInterval(interval);
    res.end();
  });
});

4. Consuming SSE Events — Three Approaches

4.1 Browser Native (EventSource API)

Works in all modern browsers with zero dependencies:

const es = new EventSource("http://localhost:4000/stream-content");

es.onopen = () => console.log("Connection established");

es.onmessage = (event) => {
  // event.data is always a string
  try {
    const parsed = JSON.parse(event.data);
    console.log("Received:", parsed);
  } catch {
    console.log("Raw string:", event.data);
  }
};

es.onerror = (err) => {
  // EventSource AUTOMATICALLY reconnects — onerror is informational only
  console.warn("SSE error (will reconnect):", err);
};

// Shutdown when done
// es.close();

4.2 Node.js Client — eventsource npm Package

For a Node.js backend that needs to consume another server's SSE stream:

npm install eventsource
import EventSource from "eventsource";

const es = new EventSource("https://api.example.com/events", {
  headers: {
    Authorization: "Bearer YOUR_TOKEN",
    "X-Client-ID": "my-service",
  },
});

es.onopen = () => console.log("Stream connected");

es.onmessage = (event) => {
  const data = JSON.parse(event.data);
  console.log("Received:", data);
};

es.addEventListener("update", (event) => {
  console.log("Named event 'update':", event.data);
});

es.onerror = (err) => {
  if (es.readyState === EventSource.CLOSED) {
    console.error("Connection closed permanently");
  }
};

4.3 Native fetch + Web Streams (Node 18+ / Browser)

No external dependencies. Ideal when you need fine-grained control:

async function consumeSSENatively(url, token) {
  const response = await fetch(url, {
    headers: {
      Authorization: `Bearer ${token}`,
      Accept: "text/event-stream",
    },
  });

  if (!response.ok || !response.body) {
    throw new Error(`HTTP ${response.status}`);
  }

  const reader = response.body
    .pipeThrough(new TextDecoderStream())
    .getReader();

  let buffer = "";

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;

    buffer += value;
    const lines = buffer.split("\n");
    buffer = lines.pop() ?? ""; // keep last incomplete line

    for (const line of lines) {
      const trimmed = line.trim();
      if (trimmed.startsWith("data:")) {
        const raw = trimmed.replace(/^data:\s*/, "");
        try {
          console.log("Parsed:", JSON.parse(raw));
        } catch {
          console.log("Raw:", raw);
        }
      }
    }
  }
}

Why the buffer trick? TCP delivers data in arbitrary chunks. A single reader.read() call may give you half an event block. The buffer accumulates bytes until a complete \n-terminated line exists before parsing.

Feature Browser EventSource eventsource npm Native fetch
Auto-reconnect ✅ Built-in ✅ Built-in ❌ Must write your own
Named events addEventListener addEventListener ❌ Must parse event: line manually
Custom headers ❌ Not supported ✅ Supported ✅ Supported
Auth tokens ❌ (use cookies/query param)
Dependencies None 1 package None
Node.js support ✅ (v18+)

5. Forwarding: Server-to-Server SSE Proxy

A real-world pattern: your backend subscribes to an upstream SSE stream and fans it out to many browser clients. This is a SSE proxy.

import express from "express";
import EventSource from "eventsource";

const app = express();

// Registry of connected browser clients
const clients = new Set();

// 1. Connect once to the upstream SSE source
const upstream = new EventSource("https://upstream-api.com/events", {
  headers: { Authorization: "Bearer UPSTREAM_TOKEN" },
});

upstream.onmessage = (event) => {
  // 2. Fan out every upstream event to all connected clients
  const message = `data: ${event.data}\n\n`;
  for (const client of clients) {
    client.write(message);
  }
};

upstream.onerror = (err) => {
  console.error("Upstream SSE error:", err);
};

// 3. Expose the fan-out endpoint to browsers
app.get("/events", (req, res) => {
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.setHeader("Connection", "keep-alive");

  clients.add(res);
  console.log(`Client connected. Total: ${clients.size}`);

  req.on("close", () => {
    clients.delete(res);
    console.log(`Client disconnected. Total: ${clients.size}`);
  });
});

app.listen(3000);

Key insight: The upstream connection is created once at startup, not per browser client. This prevents N clients from opening N upstream connections — a common and expensive bug.


6. Debugging Checklist

When events are not logging in your client, work through this list in order:

Step 1 — Verify the server with curl

# Linux / macOS
curl -N -v http://localhost:4000/stream-content

# Windows PowerShell (must use curl.exe, not the built-in alias)
curl.exe -N -v http://localhost:4000/stream-content

Why -N? This disables curl's internal output buffer. Without it, curl queues all received bytes until the connection closes — your terminal stays blank even though the server is sending data perfectly.

Expected output:

< HTTP/1.1 200 OK
< Content-Type: text/event-stream
< Cache-Control: no-cache
<
data: chunk object 1

data: chunk object 2
  • If data streams here but not in your app → the server is fine; the problem is in your client code (wrong event name, JSON parse crash, CORS).

  • If the terminal stays blank → the server is not writing, or a proxy is buffering.

Step 2 — Event name mismatch

// Server sends:
res.write(`event: priceUpdate\ndata: {"p":100}\n\n`);

// Client must use addEventListener, NOT onmessage:
es.addEventListener("priceUpdate", handler);  // ✅
es.onmessage = handler;                        // ❌ never fires for named events

Step 3 — Silent JSON parse crash

// Server sends plain text: "chunk object 1"
// Client tries:
const parsed = JSON.parse(event.data); // ❌ SyntaxError — crashes silently

// Fix: always guard
es.onmessage = (event) => {
  try {
    const parsed = JSON.parse(event.data);
    console.log(parsed);
  } catch {
    console.log("Raw:", event.data); // ✅ logs the plain string
  }
};

Step 4 — CORS blocking

If your browser client is on localhost:3000 and your server is on localhost:4000, the browser blocks the request unless the server sends CORS headers:

Access-Control-Allow-Origin: http://localhost:3000

See Section 7 for the full fix.

Step 5 — HTTP/1.1 browser tab limit

Browsers cap persistent connections per origin at 6 over HTTP/1.1. If you have 6+ tabs open to the same SSE endpoint, any new tab will open the EventSource, appear to connect (no error), but receive zero events.

Fix: Use HTTP/2 (which multiplexes unlimited streams over one TCP connection), or close extra tabs during development.


7. CORS Configuration

npm install cors
import express from "express";
import cors from "cors";

const app = express();

// Allow any origin (development only)
app.use(cors());

// Production: restrict to your frontend domain
app.use(cors({
  origin: "https://your-frontend.com",
  methods: ["GET"],
}));

app.get("/stream-content", (req, res) => {
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.setHeader("Connection", "keep-alive");

  // ... your event loop
});

Note: The cors() middleware must come before your route handlers, not after.

If your client passes an Authorization header (which browser EventSource cannot do — use the eventsource npm package or fetch instead), you also need:

app.use(cors({
  origin: "https://your-frontend.com",
  allowedHeaders: ["Authorization"],
}));

8. Buffering & Proxy Pitfalls

8.1 Express compression Middleware

If you use the compression package, it buffers output waiting for enough data to compress efficiently. Your events get stuck in that buffer.

Fix: manually flush after each write:

import compression from "compression";

app.use(compression());

app.get("/stream-content", (req, res) => {
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.setHeader("Connection", "keep-alive");

  const interval = setInterval(() => {
    res.write(`data: ping\n\n`);
    if (typeof res.flush === "function") res.flush(); // ← force flush
  }, 1000);

  req.on("close", () => {
    clearInterval(interval);
    res.end();
  });
});

8.2 Nginx Buffering

Nginx buffers proxy responses by default. Add these headers from your Express server to opt out:

res.setHeader("Cache-Control", "no-cache, no-transform");
res.setHeader("X-Accel-Buffering", "no"); // disables nginx proxy_buffering

Or configure Nginx directly:

location /stream-content {
  proxy_pass http://localhost:4000;
  proxy_buffering off;
  proxy_cache off;
  proxy_set_header Connection '';
  proxy_http_version 1.1;
  chunked_transfer_encoding on;
}

8.3 Cloudflare and CDN Layers

Cloudflare buffers responses shorter than a few KB before forwarding to clients. SSE events are tiny — they'll be held indefinitely.

Fix: Disable Cloudflare's cache for your SSE route using a Page Rule that sets Cache Level: Bypass, or deploy your SSE endpoint on a subdomain that bypasses the CDN (orange-cloud off in Cloudflare DNS).


9. Windows / PowerShell Quirks

Windows PowerShell has a built-in alias curl that points to Invoke-WebRequest — Microsoft's own cmdlet — which does not understand Unix flags like -N or -v.

# ❌ Fails — PowerShell intercepts this as Invoke-WebRequest
curl -N http://localhost:4000/stream-content

# ✅ Force the real curl executable
curl.exe -N -v http://localhost:4000/stream-content

Alternatively, check if the built-in curl alias is shadowing:

Get-Command curl
# If this returns "Alias" → you're using Invoke-WebRequest
# If it returns "Application" pointing to curl.exe → you're fine

You can also remove the alias permanently in your PowerShell profile:

Remove-Item Alias:curl -Force

10. Architecture Patterns

Pattern A — Direct Streaming (Simple Apps)

Browser → Express SSE Server → setInterval / DB query

Best for: dashboards, log tails, progress indicators. No extra infrastructure.

Pattern B — Message Queue Fan-Out (Scalable)

                        ┌─────── Browser client 1
Producer → Redis/MQ → SSE Server ─── Browser client 2
                        └─────── Browser client 3

Your Express server subscribes to a Redis pub/sub channel or a message queue (BullMQ, RabbitMQ). When a message arrives, it broadcasts to all connected SSE clients. This lets you scale horizontally — multiple SSE server instances all subscribe to the same queue.

import { createClient } from "redis";

const subscriber = createClient();
await subscriber.connect();

await subscriber.subscribe("events", (message) => {
  for (const client of clients) {
    client.write(`data: ${message}\n\n`);
  }
});

Pattern C — SSE Proxy (Aggregation)

Upstream SSE API → Your Node.js SSE Proxy → Many browsers

See Section 5. Use when you need to add auth, filter events, enrich payloads, or rate-limit before forwarding to clients.


11. SSE vs WebSockets vs Long Polling

Dimension SSE WebSockets Long Polling
Direction Server → Client only Bidirectional Server → Client (inefficient)
Protocol Plain HTTP/1.1 or HTTP/2 HTTP Upgrade → WS Plain HTTP
Auto-reconnect ✅ Built into browser ❌ Must implement ❌ Must implement
Binary data
Proxy/firewall friendly ✅ High ⚠️ Moderate ✅ High
Setup complexity Low Medium Low
Browser support All modern All modern All
Multiplexed streams ✅ With HTTP/2 Per connection
Good for Feeds, notifications, logs, AI streaming Chat, gaming, collaborative editing Legacy fallback

Decision rule:

  • Need push only + want simplicity? → SSE

  • Need two-way real-time + binary? → WebSockets

  • Legacy browser support or strict proxy env? → Long Polling


12. SSE as the Backbone of LLM Token Streaming

Every time you see an AI assistant typing out its response word by word — that is SSE. It's not a UI trick. The model genuinely cannot produce the full answer at once: it generates one token per forward pass through its neural network, and SSE is what carries each token to your screen the instant it exists.

Why LLMs can't "wait and send all at once"

A large language model is autoregressive — each token it produces becomes part of the input context used to generate the next one. There is no completed answer to send until the last token is generated. Waiting for completion before responding means the user stares at a blank screen for 10–30 seconds on a long output. SSE solves this by making latency invisible: the first token arrives in ~300ms, and the rest stream in continuously.

What the actual SSE wire looks like

The Anthropic API (and OpenAI, Mistral, Gemini — all major LLM providers) sends each token as a JSON-encoded SSE event. A real Anthropic streaming response on the wire:

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Hello"}}
 
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"!"}}
 
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"}}
 
event: message_stop
data: {"type":"message_stop"}

OpenAI-compatible APIs use a simpler format:

data: {"choices":[{"delta":{"content":"Hello"},"index":0}]}
 
data: {"choices":[{"delta":{"content":"!"},"index":0}]}
 
data: [DONE]

The [DONE] sentinel: OpenAI-style APIs terminate streams with data: [DONE] — not valid JSON. Your parser must check for this before calling JSON.parse(), otherwise you get a silent crash and miss the end-of-stream signal.

Consuming the Anthropic streaming API

import Anthropic from "@anthropic-ai/sdk";
 
const client = new Anthropic();
 
const stream = await client.messages.stream({
  model: "claude-sonnet-4-6",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Explain SSE in one paragraph." }],
});
 
// Method 1: Higher-level helper (recommended)
for await (const chunk of stream) {
  if (
    chunk.type === "content_block_delta" &&
    chunk.delta.type === "text_delta"
  ) {
    process.stdout.write(chunk.delta.text);
  }
}

The full proxy pattern: Anthropic API → your server → browser

Your Express server acts as an authenticated SSE proxy, keeping your API key server-side while streaming tokens directly to browser clients.

import express from "express";
import Anthropic from "@anthropic-ai/sdk";
 
const app = express();
app.use(express.json());
 
const client = new Anthropic(); // reads ANTHROPIC_API_KEY from env
 
app.post("/chat", async (req, res) => {
  const { message } = req.body;
 
  // Step 1: Set SSE headers on your outgoing response
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.setHeader("Connection", "keep-alive");
  res.setHeader("X-Accel-Buffering", "no"); // disable Nginx buffering
 
  try {
    const stream = client.messages.stream({
      model: "claude-sonnet-4-6",
      max_tokens: 1024,
      messages: [{ role: "user", content: message }],
    });
 
    // Step 2: Forward each token as its own SSE event
    stream.on("text", (text) => {
      res.write(`data: ${JSON.stringify({ token: text })}\n\n`);
      if (res.flush) res.flush();
    });
 
    // Step 3: Signal end of stream
    stream.on("finalMessage", () => {
      res.write("data: [DONE]\n\n");
      res.end();
    });
 
    stream.on("error", (err) => {
      res.write(`data: ${JSON.stringify({ error: err.message })}\n\n`);
      res.end();
    });
 
  } catch (err) {
    res.write(`data: ${JSON.stringify({ error: "Stream failed" })}\n\n`);
    res.end();
  }
});
 
app.listen(4000);
// Browser client
const es = new EventSource("/chat-stream");
 
es.onmessage = (event) => {
  if (event.data === "[DONE]") {
    es.close();
    hideCursor();
    return;
  }
  const { token } = JSON.parse(event.data);
  appendTokenToUI(token);
};

POST vs GET: Browser EventSource only supports GET. For chat interfaces that need to send a message body, use fetch with a streaming reader, or the @microsoft/fetch-event-source library.

Why SSE beats WebSockets for LLM streaming

Concern SSE WebSocket
Setup complexity 3 headers + res.write Upgrade handshake, ws library, message framing
CDN / proxy support Works with Cloudflare, Nginx (with config) Requires special proxy config
Auto-reconnect Built into browser Must implement manually
Data direction Server → client only (correct for LLM output) Bidirectional (unnecessary complexity)
HTTP/2 multiplexing Many streams over one TCP connection One connection per stream
Adoption Anthropic, OpenAI, Mistral, Gemini, Cohere Rare for pure token streaming

Key implementation details at production scale

✅ Check data === "[DONE]" before JSON.parse() — it is not valid JSON
✅ Filter by event: field — only content_block_delta carries text tokens
✅ Buffer partial JSON lines in a string before parsing — TCP may split event blocks
✅ Set X-Accel-Buffering: no when behind Nginx — prevents token batching
✅ Handle onerror and show a "reconnecting…" state — mid-stream drops are common
✅ Check stop_reason — max_tokens truncation looks identical to normal completion

Quick Reference

Server checklist

✅ res.setHeader("Content-Type", "text/event-stream")
✅ res.setHeader("Cache-Control", "no-cache")
✅ res.setHeader("Connection", "keep-alive")
✅ res.write(`data: payload\n\n`)   ← two newlines mandatory
✅ req.on("close", cleanup)
✅ if (res.flush) res.flush()       ← if compression middleware is used
✅ res.setHeader("X-Accel-Buffering", "no")  ← if behind Nginx

Client checklist

✅ new EventSource(url)  for browser / eventsource pkg for Node
✅ es.onmessage for unnamed events
✅ es.addEventListener("name", handler) for named events
✅ JSON.parse inside try/catch
✅ es.onerror to detect connection issues
✅ es.close() on component unmount

Debugging commands

# Test the raw SSE stream (skip the client entirely)
curl.exe -N -v http://localhost:4000/stream-content   # Windows
curl -N -v http://localhost:4000/stream-content        # macOS/Linux