Server Sent Events(SSE): The Backbone of LLM Token Streaming

From first principles to production-grade architecture.
Covers: protocol internals, Express server setup, multi-client forwarding, debugging, CORS, buffering pitfalls, and Windows quirks.
1. What Is SSE and When Should You Use It?
Server-Sent Events (SSE) is an HTTP/1.1 feature that lets a server push a continuous stream of UTF-8 text events to a client over a single, persistent connection — without the client polling.
The browser exposes this through the EventSource API. Once you new EventSource('/endpoint'), the browser fires onmessage every time the server calls res.write(...).
Mental model
Use SSE when:
You need server → client push only (live feeds, notifications, progress bars, logs).
You want automatic reconnection for free (browser EventSource handles it).
You need something simpler than WebSockets — SSE rides plain HTTP, works through corporate proxies, requires no upgrade handshake, and doesn't need a special library on the client.
Do NOT use SSE when:
You need bidirectional real-time communication → use WebSockets.
You need to push binary data (images, audio) → use WebSockets or chunked HTTP.
You have >6 tabs open on the same domain over HTTP/1.1 (browser cap) → upgrade to HTTP/2 or use WebSockets.
2. The SSE Protocol Internals
SSE is dead simple — it's plain text over a persistent HTTP connection. You only need to know four things:
2.1 Required Response Headers
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive
These tell the browser: "Don't buffer this, don't cache it, and keep the TCP socket open."
2.2 Event Block Format
Every event is one or more field lines, terminated by a blank line (\n\n):
data: your payload here\n\n
Multi-line payloads:
data: line one\n
data: line two\n\n
Named events (the client must use addEventListener, not onmessage):
event: update\n
data: {"status":"ok"}\n\n
Event IDs (allow the client to resume from a specific point on reconnect):
id: 42\n
data: {"msg":"checkpoint"}\n\n
Retry hint (tells the browser how long to wait before reconnecting, in ms):
retry: 3000\n
data: {"msg":"reconnect in 3s"}\n\n
2.3 Strict Whitespace Rules
| Rule | Correct | Broken |
|---|---|---|
| Space after colon | data: payload |
data:payload |
| Double newline to end block | ...\n\n |
...\n |
| No binary content | UTF-8 strings only | Buffer objects |
2.4 The Last-Event-ID Header
On reconnect, the browser automatically sends:
Last-Event-ID: 42
Your server can read req.headers['last-event-id'] and resume streaming from that point — enabling lossless reconnection.
3. Building the SSE Server (Express)
3.1 Minimal Working Server
import express from "express";
const app = express();
app.get("/stream-content", (req, res) => {
// Step 1: Set the three mandatory SSE headers
res.setHeader("Content-Type", "text/event-stream");
res.setHeader("Cache-Control", "no-cache");
res.setHeader("Connection", "keep-alive");
let counter = 1;
// Step 2: Push an event every second
const interval = setInterval(() => {
res.write(`data: chunk object ${counter++}\n\n`);
}, 1000);
// Step 3: Clean up when the client disconnects
req.on("close", () => {
clearInterval(interval);
res.end();
});
});
app.listen(4000, () => console.log("SSE Server running on port 4000"));
Why req.on('close') not res.on('close')?req represents the incoming connection. When the browser navigates away or closes the tab, Node.js emits close on the request object. Using req.on('close') is the canonical pattern — always clean up your timers and database subscriptions here to prevent memory leaks.
3.2 Sending Structured JSON Events
const interval = setInterval(() => {
const payload = JSON.stringify({
timestamp: new Date().toISOString(),
value: Math.random() * 100,
});
res.write(`data: ${payload}\n\n`);
}, 1000);
On the client:
es.onmessage = (event) => {
const { timestamp, value } = JSON.parse(event.data);
console.log(timestamp, value);
};
3.3 Using Named Events
// Server
res.write(`event: priceUpdate\ndata: {"symbol":"BTC","price":67000}\n\n`);
res.write(`event: alert\ndata: {"level":"warn","msg":"High load"}\n\n`);
// Client — onmessage will NOT fire for named events
es.addEventListener("priceUpdate", (e) => {
const { symbol, price } = JSON.parse(e.data);
});
es.addEventListener("alert", (e) => {
const { level, msg } = JSON.parse(e.data);
});
Common trap: If your server sends
event: update\n, but your client only hases.onmessage, nothing logs.onmessageonly fires for unnamed (default) events.
3.4 Resumable Streams with Event IDs
app.get("/stream-content", (req, res) => {
res.setHeader("Content-Type", "text/event-stream");
res.setHeader("Cache-Control", "no-cache");
res.setHeader("Connection", "keep-alive");
// Check if client is resuming
const lastId = parseInt(req.headers["last-event-id"] ?? "0", 10);
let counter = lastId + 1;
const interval = setInterval(() => {
res.write(`id: \({counter}\ndata: event \){counter}\n\n`);
counter++;
}, 1000);
req.on("close", () => {
clearInterval(interval);
res.end();
});
});
4. Consuming SSE Events — Three Approaches
4.1 Browser Native (EventSource API)
Works in all modern browsers with zero dependencies:
const es = new EventSource("http://localhost:4000/stream-content");
es.onopen = () => console.log("Connection established");
es.onmessage = (event) => {
// event.data is always a string
try {
const parsed = JSON.parse(event.data);
console.log("Received:", parsed);
} catch {
console.log("Raw string:", event.data);
}
};
es.onerror = (err) => {
// EventSource AUTOMATICALLY reconnects — onerror is informational only
console.warn("SSE error (will reconnect):", err);
};
// Shutdown when done
// es.close();
4.2 Node.js Client — eventsource npm Package
For a Node.js backend that needs to consume another server's SSE stream:
npm install eventsource
import EventSource from "eventsource";
const es = new EventSource("https://api.example.com/events", {
headers: {
Authorization: "Bearer YOUR_TOKEN",
"X-Client-ID": "my-service",
},
});
es.onopen = () => console.log("Stream connected");
es.onmessage = (event) => {
const data = JSON.parse(event.data);
console.log("Received:", data);
};
es.addEventListener("update", (event) => {
console.log("Named event 'update':", event.data);
});
es.onerror = (err) => {
if (es.readyState === EventSource.CLOSED) {
console.error("Connection closed permanently");
}
};
4.3 Native fetch + Web Streams (Node 18+ / Browser)
No external dependencies. Ideal when you need fine-grained control:
async function consumeSSENatively(url, token) {
const response = await fetch(url, {
headers: {
Authorization: `Bearer ${token}`,
Accept: "text/event-stream",
},
});
if (!response.ok || !response.body) {
throw new Error(`HTTP ${response.status}`);
}
const reader = response.body
.pipeThrough(new TextDecoderStream())
.getReader();
let buffer = "";
while (true) {
const { value, done } = await reader.read();
if (done) break;
buffer += value;
const lines = buffer.split("\n");
buffer = lines.pop() ?? ""; // keep last incomplete line
for (const line of lines) {
const trimmed = line.trim();
if (trimmed.startsWith("data:")) {
const raw = trimmed.replace(/^data:\s*/, "");
try {
console.log("Parsed:", JSON.parse(raw));
} catch {
console.log("Raw:", raw);
}
}
}
}
}
Why the buffer trick? TCP delivers data in arbitrary chunks. A single reader.read() call may give you half an event block. The buffer accumulates bytes until a complete \n-terminated line exists before parsing.
| Feature | Browser EventSource | eventsource npm |
Native fetch |
|---|---|---|---|
| Auto-reconnect | ✅ Built-in | ✅ Built-in | ❌ Must write your own |
| Named events | ✅ addEventListener |
✅ addEventListener |
❌ Must parse event: line manually |
| Custom headers | ❌ Not supported | ✅ Supported | ✅ Supported |
| Auth tokens | ❌ (use cookies/query param) | ✅ | ✅ |
| Dependencies | None | 1 package | None |
| Node.js support | ❌ | ✅ | ✅ (v18+) |
5. Forwarding: Server-to-Server SSE Proxy
A real-world pattern: your backend subscribes to an upstream SSE stream and fans it out to many browser clients. This is a SSE proxy.
import express from "express";
import EventSource from "eventsource";
const app = express();
// Registry of connected browser clients
const clients = new Set();
// 1. Connect once to the upstream SSE source
const upstream = new EventSource("https://upstream-api.com/events", {
headers: { Authorization: "Bearer UPSTREAM_TOKEN" },
});
upstream.onmessage = (event) => {
// 2. Fan out every upstream event to all connected clients
const message = `data: ${event.data}\n\n`;
for (const client of clients) {
client.write(message);
}
};
upstream.onerror = (err) => {
console.error("Upstream SSE error:", err);
};
// 3. Expose the fan-out endpoint to browsers
app.get("/events", (req, res) => {
res.setHeader("Content-Type", "text/event-stream");
res.setHeader("Cache-Control", "no-cache");
res.setHeader("Connection", "keep-alive");
clients.add(res);
console.log(`Client connected. Total: ${clients.size}`);
req.on("close", () => {
clients.delete(res);
console.log(`Client disconnected. Total: ${clients.size}`);
});
});
app.listen(3000);
Key insight: The upstream connection is created once at startup, not per browser client. This prevents N clients from opening N upstream connections — a common and expensive bug.
6. Debugging Checklist
When events are not logging in your client, work through this list in order:
Step 1 — Verify the server with curl
# Linux / macOS
curl -N -v http://localhost:4000/stream-content
# Windows PowerShell (must use curl.exe, not the built-in alias)
curl.exe -N -v http://localhost:4000/stream-content
Why -N? This disables curl's internal output buffer. Without it, curl queues all received bytes until the connection closes — your terminal stays blank even though the server is sending data perfectly.
Expected output:
< HTTP/1.1 200 OK
< Content-Type: text/event-stream
< Cache-Control: no-cache
<
data: chunk object 1
data: chunk object 2
If data streams here but not in your app → the server is fine; the problem is in your client code (wrong event name, JSON parse crash, CORS).
If the terminal stays blank → the server is not writing, or a proxy is buffering.
Step 2 — Event name mismatch
// Server sends:
res.write(`event: priceUpdate\ndata: {"p":100}\n\n`);
// Client must use addEventListener, NOT onmessage:
es.addEventListener("priceUpdate", handler); // ✅
es.onmessage = handler; // ❌ never fires for named events
Step 3 — Silent JSON parse crash
// Server sends plain text: "chunk object 1"
// Client tries:
const parsed = JSON.parse(event.data); // ❌ SyntaxError — crashes silently
// Fix: always guard
es.onmessage = (event) => {
try {
const parsed = JSON.parse(event.data);
console.log(parsed);
} catch {
console.log("Raw:", event.data); // ✅ logs the plain string
}
};
Step 4 — CORS blocking
If your browser client is on localhost:3000 and your server is on localhost:4000, the browser blocks the request unless the server sends CORS headers:
Access-Control-Allow-Origin: http://localhost:3000
See Section 7 for the full fix.
Step 5 — HTTP/1.1 browser tab limit
Browsers cap persistent connections per origin at 6 over HTTP/1.1. If you have 6+ tabs open to the same SSE endpoint, any new tab will open the EventSource, appear to connect (no error), but receive zero events.
Fix: Use HTTP/2 (which multiplexes unlimited streams over one TCP connection), or close extra tabs during development.
7. CORS Configuration
npm install cors
import express from "express";
import cors from "cors";
const app = express();
// Allow any origin (development only)
app.use(cors());
// Production: restrict to your frontend domain
app.use(cors({
origin: "https://your-frontend.com",
methods: ["GET"],
}));
app.get("/stream-content", (req, res) => {
res.setHeader("Content-Type", "text/event-stream");
res.setHeader("Cache-Control", "no-cache");
res.setHeader("Connection", "keep-alive");
// ... your event loop
});
Note: The
cors()middleware must come before your route handlers, not after.
If your client passes an Authorization header (which browser EventSource cannot do — use the eventsource npm package or fetch instead), you also need:
app.use(cors({
origin: "https://your-frontend.com",
allowedHeaders: ["Authorization"],
}));
8. Buffering & Proxy Pitfalls
8.1 Express compression Middleware
If you use the compression package, it buffers output waiting for enough data to compress efficiently. Your events get stuck in that buffer.
Fix: manually flush after each write:
import compression from "compression";
app.use(compression());
app.get("/stream-content", (req, res) => {
res.setHeader("Content-Type", "text/event-stream");
res.setHeader("Cache-Control", "no-cache");
res.setHeader("Connection", "keep-alive");
const interval = setInterval(() => {
res.write(`data: ping\n\n`);
if (typeof res.flush === "function") res.flush(); // ← force flush
}, 1000);
req.on("close", () => {
clearInterval(interval);
res.end();
});
});
8.2 Nginx Buffering
Nginx buffers proxy responses by default. Add these headers from your Express server to opt out:
res.setHeader("Cache-Control", "no-cache, no-transform");
res.setHeader("X-Accel-Buffering", "no"); // disables nginx proxy_buffering
Or configure Nginx directly:
location /stream-content {
proxy_pass http://localhost:4000;
proxy_buffering off;
proxy_cache off;
proxy_set_header Connection '';
proxy_http_version 1.1;
chunked_transfer_encoding on;
}
8.3 Cloudflare and CDN Layers
Cloudflare buffers responses shorter than a few KB before forwarding to clients. SSE events are tiny — they'll be held indefinitely.
Fix: Disable Cloudflare's cache for your SSE route using a Page Rule that sets Cache Level: Bypass, or deploy your SSE endpoint on a subdomain that bypasses the CDN (orange-cloud off in Cloudflare DNS).
9. Windows / PowerShell Quirks
Windows PowerShell has a built-in alias curl that points to Invoke-WebRequest — Microsoft's own cmdlet — which does not understand Unix flags like -N or -v.
# ❌ Fails — PowerShell intercepts this as Invoke-WebRequest
curl -N http://localhost:4000/stream-content
# ✅ Force the real curl executable
curl.exe -N -v http://localhost:4000/stream-content
Alternatively, check if the built-in curl alias is shadowing:
Get-Command curl
# If this returns "Alias" → you're using Invoke-WebRequest
# If it returns "Application" pointing to curl.exe → you're fine
You can also remove the alias permanently in your PowerShell profile:
Remove-Item Alias:curl -Force
10. Architecture Patterns
Pattern A — Direct Streaming (Simple Apps)
Browser → Express SSE Server → setInterval / DB query
Best for: dashboards, log tails, progress indicators. No extra infrastructure.
Pattern B — Message Queue Fan-Out (Scalable)
┌─────── Browser client 1
Producer → Redis/MQ → SSE Server ─── Browser client 2
└─────── Browser client 3
Your Express server subscribes to a Redis pub/sub channel or a message queue (BullMQ, RabbitMQ). When a message arrives, it broadcasts to all connected SSE clients. This lets you scale horizontally — multiple SSE server instances all subscribe to the same queue.
import { createClient } from "redis";
const subscriber = createClient();
await subscriber.connect();
await subscriber.subscribe("events", (message) => {
for (const client of clients) {
client.write(`data: ${message}\n\n`);
}
});
Pattern C — SSE Proxy (Aggregation)
Upstream SSE API → Your Node.js SSE Proxy → Many browsers
See Section 5. Use when you need to add auth, filter events, enrich payloads, or rate-limit before forwarding to clients.
11. SSE vs WebSockets vs Long Polling
| Dimension | SSE | WebSockets | Long Polling |
|---|---|---|---|
| Direction | Server → Client only | Bidirectional | Server → Client (inefficient) |
| Protocol | Plain HTTP/1.1 or HTTP/2 | HTTP Upgrade → WS | Plain HTTP |
| Auto-reconnect | ✅ Built into browser | ❌ Must implement | ❌ Must implement |
| Binary data | ❌ | ✅ | ❌ |
| Proxy/firewall friendly | ✅ High | ⚠️ Moderate | ✅ High |
| Setup complexity | Low | Medium | Low |
| Browser support | All modern | All modern | All |
| Multiplexed streams | ✅ With HTTP/2 | Per connection | ❌ |
| Good for | Feeds, notifications, logs, AI streaming | Chat, gaming, collaborative editing | Legacy fallback |
Decision rule:
Need push only + want simplicity? → SSE
Need two-way real-time + binary? → WebSockets
Legacy browser support or strict proxy env? → Long Polling
12. SSE as the Backbone of LLM Token Streaming
Every time you see an AI assistant typing out its response word by word — that is SSE. It's not a UI trick. The model genuinely cannot produce the full answer at once: it generates one token per forward pass through its neural network, and SSE is what carries each token to your screen the instant it exists.
Why LLMs can't "wait and send all at once"
A large language model is autoregressive — each token it produces becomes part of the input context used to generate the next one. There is no completed answer to send until the last token is generated. Waiting for completion before responding means the user stares at a blank screen for 10–30 seconds on a long output. SSE solves this by making latency invisible: the first token arrives in ~300ms, and the rest stream in continuously.
What the actual SSE wire looks like
The Anthropic API (and OpenAI, Mistral, Gemini — all major LLM providers) sends each token as a JSON-encoded SSE event. A real Anthropic streaming response on the wire:
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Hello"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"!"}}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"}}
event: message_stop
data: {"type":"message_stop"}
OpenAI-compatible APIs use a simpler format:
data: {"choices":[{"delta":{"content":"Hello"},"index":0}]}
data: {"choices":[{"delta":{"content":"!"},"index":0}]}
data: [DONE]
The [DONE] sentinel: OpenAI-style APIs terminate streams with
data: [DONE]— not valid JSON. Your parser must check for this before callingJSON.parse(), otherwise you get a silent crash and miss the end-of-stream signal.
Consuming the Anthropic streaming API
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const stream = await client.messages.stream({
model: "claude-sonnet-4-6",
max_tokens: 1024,
messages: [{ role: "user", content: "Explain SSE in one paragraph." }],
});
// Method 1: Higher-level helper (recommended)
for await (const chunk of stream) {
if (
chunk.type === "content_block_delta" &&
chunk.delta.type === "text_delta"
) {
process.stdout.write(chunk.delta.text);
}
}
The full proxy pattern: Anthropic API → your server → browser
Your Express server acts as an authenticated SSE proxy, keeping your API key server-side while streaming tokens directly to browser clients.
import express from "express";
import Anthropic from "@anthropic-ai/sdk";
const app = express();
app.use(express.json());
const client = new Anthropic(); // reads ANTHROPIC_API_KEY from env
app.post("/chat", async (req, res) => {
const { message } = req.body;
// Step 1: Set SSE headers on your outgoing response
res.setHeader("Content-Type", "text/event-stream");
res.setHeader("Cache-Control", "no-cache");
res.setHeader("Connection", "keep-alive");
res.setHeader("X-Accel-Buffering", "no"); // disable Nginx buffering
try {
const stream = client.messages.stream({
model: "claude-sonnet-4-6",
max_tokens: 1024,
messages: [{ role: "user", content: message }],
});
// Step 2: Forward each token as its own SSE event
stream.on("text", (text) => {
res.write(`data: ${JSON.stringify({ token: text })}\n\n`);
if (res.flush) res.flush();
});
// Step 3: Signal end of stream
stream.on("finalMessage", () => {
res.write("data: [DONE]\n\n");
res.end();
});
stream.on("error", (err) => {
res.write(`data: ${JSON.stringify({ error: err.message })}\n\n`);
res.end();
});
} catch (err) {
res.write(`data: ${JSON.stringify({ error: "Stream failed" })}\n\n`);
res.end();
}
});
app.listen(4000);
// Browser client
const es = new EventSource("/chat-stream");
es.onmessage = (event) => {
if (event.data === "[DONE]") {
es.close();
hideCursor();
return;
}
const { token } = JSON.parse(event.data);
appendTokenToUI(token);
};
POST vs GET: Browser
EventSourceonly supports GET. For chat interfaces that need to send a message body, usefetchwith a streaming reader, or the@microsoft/fetch-event-sourcelibrary.
Why SSE beats WebSockets for LLM streaming
| Concern | SSE | WebSocket |
|---|---|---|
| Setup complexity | 3 headers + res.write | Upgrade handshake, ws library, message framing |
| CDN / proxy support | Works with Cloudflare, Nginx (with config) | Requires special proxy config |
| Auto-reconnect | Built into browser | Must implement manually |
| Data direction | Server → client only (correct for LLM output) | Bidirectional (unnecessary complexity) |
| HTTP/2 multiplexing | Many streams over one TCP connection | One connection per stream |
| Adoption | Anthropic, OpenAI, Mistral, Gemini, Cohere | Rare for pure token streaming |
Key implementation details at production scale
✅ Check data === "[DONE]" before JSON.parse() — it is not valid JSON
✅ Filter by event: field — only content_block_delta carries text tokens
✅ Buffer partial JSON lines in a string before parsing — TCP may split event blocks
✅ Set X-Accel-Buffering: no when behind Nginx — prevents token batching
✅ Handle onerror and show a "reconnecting…" state — mid-stream drops are common
✅ Check stop_reason — max_tokens truncation looks identical to normal completion
Quick Reference
Server checklist
✅ res.setHeader("Content-Type", "text/event-stream")
✅ res.setHeader("Cache-Control", "no-cache")
✅ res.setHeader("Connection", "keep-alive")
✅ res.write(`data: payload\n\n`) ← two newlines mandatory
✅ req.on("close", cleanup)
✅ if (res.flush) res.flush() ← if compression middleware is used
✅ res.setHeader("X-Accel-Buffering", "no") ← if behind Nginx
Client checklist
✅ new EventSource(url) for browser / eventsource pkg for Node
✅ es.onmessage for unnamed events
✅ es.addEventListener("name", handler) for named events
✅ JSON.parse inside try/catch
✅ es.onerror to detect connection issues
✅ es.close() on component unmount
Debugging commands
# Test the raw SSE stream (skip the client entirely)
curl.exe -N -v http://localhost:4000/stream-content # Windows
curl -N -v http://localhost:4000/stream-content # macOS/Linux




