WebSockets and Polling
Four ways for a server to tell a client "something changed" — and what each costs at a million clients.
Big idea
HTTP is request–response: the server can only speak when asked. To get updates to clients quickly, you either ask repeatedly (short polling), ask and wait (long polling), or keep a connection open so the server can push (server-sent events, WebSockets). The choice trades latency and wasted requests against the cost of holding millions of open connections.
The idea
A chat message, a driver's location, a stock price, a build finishing — the server knows first, and the client wants to know as soon as possible. Plain HTTP has no way for the server to start a conversation, so each technique below is a different workaround, from "keep asking" to "never hang up".
Think of it like
Waiting for a parcel. Short polling is walking to the door every five minutes to check. Long polling is calling the courier and staying on the line until they say "it's here" (then calling again). Server-sent events is a delivery-updates radio you leave on. A WebSocket is a phone line that stays open both ways, so you can also say "leave it with the neighbour" without dialling again.
The four techniques
Short polling — the client asks every N seconds: GET /messages?since=…. Simple, works everywhere, but an update waits up to N seconds, and almost every request returns nothing.
Long polling — the client asks, and the server holds the request open until there's something to send (or a timeout, ~30 s). When it responds, the client immediately asks again. Near-instant updates over plain HTTP.
Server-sent events (SSE) — one long-lived HTTP response that the server keeps writing events into (Content-Type: text/event-stream). One direction only (server → client), automatic reconnection with a resume point (Last-Event-ID) built into browsers. Ideal for feeds, notifications, and streaming model output token by token.
WebSockets — an HTTP request that upgrades into a persistent, two-way connection (Upgrade: websocket). Either side can send a message at any time with a few bytes of framing. The choice for chat, multiplayer, collaborative editing — anything talking both ways, often.
Watch it
The same update, seen by a client that polls and a client on a WebSocket.
What it costs at a million clients
Say 1 million clients want updates within a few seconds, and a typical client gets one real update a minute.
| Technique | Requests/connections | Delay | Notes |
|---|---|---|---|
| Short poll every 5 s | 200,000 requests/s — 17 billion a day, ~92% of them empty | up to 5 s | ~160 MB/s of headers alone at ~800 bytes each |
| Long poll, 30 s timeout | ~33,000 requests/s (timeouts) + one per update | near-instant | a held request per client |
| Server-sent events | 1 million open connections | near-instant | one direction; auto-reconnect |
| WebSocket | 1 million open connections, ~33,000 small heartbeats/s | near-instant | two-way; tiny per-message overhead |
Short polling pays per request; the push options pay per connection. An idle open connection costs memory (on the order of 10–50 KB each) but almost no CPU — so a tuned server holds hundreds of thousands of them, while it could never answer 200,000 empty polls a second cheaply.
What changes when connections are long-lived
- Load balancing. A connection sticks to one server for hours. When you add servers, existing clients don't move to them — balance by connection count, and rebalance slowly by asking some clients to reconnect.
- Routing a message to a user. The sender's request lands on server A; the recipient is connected to server B. You need a registry (user → server) or a pub/sub layer (Redis, Kafka) that every connection server subscribes to. The WhatsApp case study works through this.
- Deploys. Restarting a server drops every connection on it. Drain gracefully — stop accepting new connections, ask clients to reconnect elsewhere, then exit — and make clients reconnect with randomized backoff so they don't all arrive at once.
- Missed updates. A reconnecting client may have missed events. Give every event an ID or sequence number, and let the client ask for everything after the last one it saw.
- Proxies and firewalls. Some drop idle connections after ~60 s. A heartbeat (ping) every 20–30 s keeps the connection alive and detects dead peers.
Choosing
- Updates are rare and a few seconds' delay is fine → short polling. Boring is good.
- Server → client only (notifications, dashboards, streaming AI responses) → SSE.
- Frequent two-way traffic (chat, games, live collaboration) → WebSockets.
- Some network in the path blocks the above → long polling as the fallback.
You're streaming an AI model's answer to a browser, token by token. The browser sends nothing back mid-answer. What fits best?
500,000 clients short-poll every 10 seconds. How many requests per second is that?
Key terms
- Short polling — Asking the server for updates on a fixed interval.
- Long polling — A request the server holds open until it has an update.
- Server-sent events (SSE) — A long-lived HTTP response carrying a stream of server → client events.
- WebSocket — A persistent two-way connection upgraded from an HTTP request.
- Heartbeat — A small periodic message that keeps a connection alive and detects dead peers.
- Sticky connection — A long-lived connection that stays on one server, which changes how load balancing works.