Follow-up Tree: WebSockets

Overview

“How is a WebSocket different from an HTTP request, and why is that difference a scaling problem?” is the follow-up that finds the boundary between candidates who have used a WebSocket library and candidates who have operated one. The opening answer — persistent, bidirectional, low-latency messaging over one TCP connection — is easy; every interesting consequence follows from two facts the interviewer is waiting to hear: the connection is stateful (it lives in exactly one server’s memory) and it is long-lived (it outlives deploys, reschedules, and network blips). Those two facts drive the ladder: the upgrade handshake that creates the connection, the collision between stateful connections and stateless infrastructure, the two scaling architectures (sticky sessions versus pub/sub fan-out) with their fan-out arithmetic, and the reconnect storm that turns one server restart into a thundering herd. The chain leans on neighbors for what it does not own: the HTTP mechanics of the handshake belong to HTTP Protocol Evolution, the TCP lifecycle underneath is TCP Lifecycle, and the queue mechanics behind server-side fan-out are Message Queues. This page owns the socket itself: what the upgrade really does, why every connection is a piece of state the rest of your infrastructure cannot see, and what it costs to keep thousands of them alive across deploys and failures.

The Opening Question

“What is a WebSocket, and why does it exist when HTTP already works?”

The expected core answer: a WebSocket is a protocol that starts as an HTTP request and upgrades into a persistent, full-duplex TCP connection, standardized in RFC 6455. The client sends an HTTP/1.1 GET with Upgrade: websocket and Connection: Upgrade; a server that agrees responds 101 Switching Protocols, and from that byte onward the connection no longer speaks HTTP — it speaks WebSocket framing (small headers, opcode-tagged text/binary frames, ping/pong control frames). It exists because HTTP request/response cannot let a server initiate data: a client wanting live updates must poll (wasteful, laggy) or long-poll (holds requests open, but every update costs a new request cycle). Auctions, chats, collaborative editing, live dashboards — anything where the server must push within milliseconds — needs a channel the server can write to whenever it wants.

Why this answer is correct: it locates the upgrade at the HTTP boundary and names what changes on the far side of the 101 — a different framing protocol, not “HTTP with push.” Interviewers probe exactly there (“so is it still HTTP?”), and “the first request is HTTP; after the 101 it is a different protocol over the same TCP connection” answers the question the next rung formalizes.

Level 2: The Upgrade Handshake — What Exactly Changes on the Wire?

What it tests: whether the handshake is a memorized incantation or an understood protocol transition — and whether you know why each header exists.

Weak answer: “The client sends an upgrade request and the server says yes, then they talk” — no model of the verification step, no awareness that post-101 bytes are not HTTP, no answer for a proxy that does not cooperate.

Strong answer: walks the exchange: client GET with Upgrade: websocket, Connection: Upgrade, Sec-WebSocket-Version: 13, and Sec-WebSocket-Key (a random nonce); the server responds 101 with Sec-WebSocket-Accept — SHA-1 of the key concatenated with a fixed GUID constant from the RFC — a deliberately checkable proof that the responder understands WebSocket rather than being a misconfigured cache echoing headers. Explains why the handshake rides HTTP/1.1 (universal intermediary support; Upgrade is the sanctioned escape hatch) and what that implies: every proxy in the path must allow the upgrade, and post-upgrade the connection carries WebSocket frames — 2–14 byte headers, opcodes for text/binary/ping/pong/close, and client-to-server frames masked with a random key (a defense from the proxy-cache poisoning era).

Escalation trigger: “Your WebSocket works locally and dies within seconds behind a corporate proxy. What is happening, and how do the handshake semantics explain both the failure and the fix?”

Answer. After the 101, the TCP connection goes quiet from the intermediary’s point of view: it looks like an HTTP request that never finished — no more headers, just bytes in a protocol the intermediary does not parse. A proxy that inspects traffic, or an LB with an idle timeout in tens of seconds, sees a stale half-request and resets it. The handshake semantics are why the protocol anticipated this: the 101 is the contract that the connection is exempt from HTTP request/response accounting, but the contract only binds intermediaries that honor it — the fix family is about getting bytes past them: ensure the LB supports Upgrade (hop-by-hop headers forwarded, idle timeouts raised), and add application-level ping/pong well inside every intermediary’s idle window (the RFC’s control frames exist for exactly this keepalive role, and the application heartbeat doubles as liveness detection — the difference between TCP keepalive and an app-level ping is a favorite sub-question, and TCP Lifecycle owns the kernel-side half). The deeper lesson: WebSocket’s compatibility strategy was tunnel through HTTP’s port and intermediaries, which made deployment trivial and made every HTTP-era default (idle timeouts, buffering, connection limits) a live failure mode. Framing, masking, and negotiation details are the WebSocket Protocol chapter; the practical contract is the WebSocket page.

Level 3: Stateful Connections vs Stateless Infrastructure — Why This Breaks Your Deployment

What it tests: whether you can articulate where the connection state physically lives and what that costs during failover and deploys, in contrast to the stateless HTTP world your infrastructure was built for.

Weak answer: “WebSockets are just long HTTP requests” — precisely wrong in the way that breaks load balancers, autoscalers, and rolling deploys, none designed for a “request” that lives for hours and holds a server-side subscription set.

Strong answer: states the invariant: a WebSocket connection is state (a socket fd, buffers, the authenticated identity, the subscription list) held in exactly one process on exactly one machine; the rest of the fleet cannot serve it, and the LB must keep sending everything for that connection to the same place. Contrasts with stateless HTTP, where any replica serves any request and only the backend datastore is shared — and walks the consequences: sticky routing becomes a correctness requirement, not an optimization (the Load Balancing chain’s affinity rung); a server crash severs every connection it held, and subscription state must be recoverable from a shared store or rebuilt; rolling deploys and scale-down must drain connections (stop accepting new, wait or migrate existing) or every deploy is a mass-disconnect event; and memory-per-connection becomes the capacity unit — an idle connection still holds kernel buffers and application context, so “10,000 idle clients” is a memory budget, not a traffic number.

Escalation trigger: “Every deploy of your WebSocket service disconnects all 200,000 clients for about 30 seconds. Walk through why, and design the deploy that doesn’t.”

Answer. Why: the deploy kills old processes (SIGTERM), the kernel closes their sockets, every client sees the drop and reconnects — and because reconnection is uncontrolled, the fleet absorbs a reconnect burst while in-flight subscriptions are rebuilt cold. The design that does not: drain before termination — the LB stops routing new connections to the dying instance while it keeps serving existing ones; the instance signals clients over the existing connection (“you will be moved; reconnect to a drain endpoint with a jittered delay”), converting a mass-reconnect into a scheduled, staggered migration; the server checkpoints subscription state to the shared store before the final close, so reconnects restore context instantly; and new instances warm before old ones drain, so capacity never dips. The transferable frame: stateless infrastructure treats connections as disposable because no per-request state lives in them; a WebSocket server must own a shutdown contract — the same SIGTERM discipline the Kubernetes Pod Lifecycle chain develops for pods, applied to state that cannot be re-created from the request. Where state cannot be drained in time, the honest alternative is accepting the disconnect and engineering the reconnect path deliberately — the storm rung next.

Level 4: Scaling — Sticky Sessions vs Pub/Sub Fan-Out

What it tests: whether you know the two real architectures for multi-server WebSocket fleets — and can do the fan-out arithmetic that decides between them, rather than naming tools.

Weak answer: “Use sticky sessions with a Redis for state” — a tool concatenation with no model of what each piece buys and no answer for the fan-out problem that actually kills these systems: one event, a million recipients.

Strong answer: distinguishes the architectures by where the routing decision lives. Sticky sessions: the LB pins each connection to its server; delivering to a user requires knowing which server holds them (a registry: user → instance), and every publisher must target instances individually — it works, but couples delivery to membership (a dead server strands its users until reconnection). Pub/sub fan-out: connections are state at the edge, but subscriptions are data in a broker; a publisher publishes once to a channel, and each connected server receives the event and delivers to its local subscribers — delivery becomes the broker’s problem (Message Queues owns delivery semantics; pub/sub the pattern). Then the arithmetic: a celebrity posts to one million followers — one publish, broker fans out to every server holding a follower, and the global write amplification is 1M deliveries no matter the tool, so the design question is which tier absorbs it (dedicated fan-out workers, coalescing, sharded channels). This is where naive “just publish to Redis pub/sub” dies: Redis pub/sub is fire-and-forget, so a slow or disconnected server silently loses the message, and offline followers need a different path entirely (push notification or feed-read-on-reconnect — the delivery-guarantee split is Webhook Delivery territory).

Escalation trigger: “One user with 2M followers posts. Your Redis pub/sub design melts. What actually failed, and what does the fix look like?”

Answer. Fan-out amplification met fire-and-forget delivery: the single publish produces 2M deliveries; every app server receives the event for its local subscribers (fine), but the fan-out path saturates — the broker’s outbound bandwidth, the servers’ write loops (per-connection backpressure means one slow client slows its server’s dispatch loop), and any server mid-deploy simply drops the event with no retry, because Redis pub/sub has no durability or per-consumer ack. The fix is architectural: move fan-out out of the connection servers into dedicated workers (publish once to a durable queue; a fleet of fan-out workers reads the follower list in shards and enqueues per-server or per-user deliveries, with acks and retries — Kafka as the spine); coalesce and batch deliveries per socket (one write per tick, not one per message); and split delivery classes — online followers get realtime push, offline ones get feed-read-on-reconnect. The senior framing to volunteer: scaling WebSockets is not a connection problem (per-connection cost is small, and the Virtual Threads vs Async chain covers the I/O model), it is a routing and fan-out problem — and once fan-out is the bottleneck, the answer is queueing theory and delivery semantics, not socket tuning.

Level 5: Reconnect Storms and Judgment — When the Herd Comes Home

What it tests: whether you model reconnection as a load event with its own dynamics — and can place WebSockets honestly among the alternatives (SSE, polling, QUIC migration) by workload.

Weak answer: “Clients auto-reconnect with exponential backoff” — the library default recited as a strategy: no jitter, no cap, no server-side coordination, no thought about what a server restart does to 200,000 synchronized backoff timers.

Strong answer: treats the reconnect as a thundering herd: after a fleet-wide blip, all clients retry simultaneously; without jitter they re-synchronize on every retry; the storm lands on the handshake path (TLS + upgrade is the expensive per-connection path) and can keep the service down long after the original fault cleared. Mitigations are a coordinated pair: client-side — exponential backoff with jitter (decorrelated), capped, respecting a server-provided retry hint on close; server-side — admission control on the reconnect path (a connection-rate limiter that briefly queues excess reconnects rather than accepting into thrash) and restarting instances in waves so the herd divides. Knows the storm is not only failure-driven: mobile networks churn connections constantly, so steady-state reconnect rate is a capacity line item, and heartbeat interval trades fast failure detection against battery and idle-timeout cost. Then the judgment: WebSockets are right when the server must push with low latency to a managed set of connected clients; server-sent events are strictly better for unidirectional streams (one-way, auto-reconnect built in, plain HTTP); polling is fine when data changes slowly; and QUIC changes the mobility calculus — the connection survives IP changes, eliminating a class of mobile reconnects (QUIC Connection Migration).

Escalation trigger: “A deployment gone wrong restarts all 30 instances within 40 seconds. Clients reconnect with 30 s capped exponential backoff, no jitter. Describe minute one, minute two, and the fixes.”

Answer. Minute one: at t≈0 all connections drop; clients reconnect immediately (first attempt has no delay) — hundreds of thousands of TLS handshakes and upgrades land on 30 freshly-started, cache-cold instances; the handshake path saturates; most time out and enter backoff. At t≈30 s the entire cohort retries in unison (capped backoff without jitter synchronizes every attempt), producing a second wave exactly when the first wave’s half-finished handshakes still consume capacity — a self-sustaining oscillation that outlives the deployment by far. Minute two looks like an outage with no outage cause. The fixes, by leverage: jitter every backoff (decorrelated jitter spreads the cohort across the window — the single change that dissolves the synchronized wave); admission-limit the upgrade path per instance (accept a bounded rate, reject the rest fast so they back off while demand drains); stagger the deploy into waves (never restart the fleet in one window); and warm the handshake path (resumption tickets, LB spread). The lesson closing the ladder: in a long-lived-connection system, transition events — deploys, failures, network churn — are the load peaks, and every design decision is about turning synchronized transitions into spread ones. The storm arithmetic is the Redis Caching chain’s stampede pattern wearing a handshake costume; the underlying herd dynamics are Queueing Fundamentals.

How to Approach Unfamiliar Variations

1. Find the state, find the failure. Any WebSocket variation — new transport, new broker, mobile clients — is analyzed by locating the connection state (which process, which memory) and asking what happens to it on failure, deploy, and scale-out. State location predicts every operational behavior.
2. Count the fan-out before choosing the tool. For any broadcast or presence feature, compute deliveries = event rate × recipients, and ask which tier absorbs it. Tools do not change the arithmetic; they change who retries.
3. Treat transitions as load. Deploys, failures, and network churn are connection churn, and churn synchronizes. Every variant answer includes: drain, jitter, admission-limit the handshake path.
4. Re-derive the transport choice from the data flow. Bidirectional push-latency-sensitive → WebSocket; one-way server push → SSE; slow-changing data → polling; mobile clients behind changing networks → QUIC-native paths. When the variation changes the flow shape, redo the choice instead of defending the default.

Interview Questions

Q: Why does the WebSocket handshake include the Sec-WebSocket-Accept computation at all? A: As a deliberate competence check: the client verifies the responder actually executed the RFC’s derivation (SHA-1 of key + fixed GUID) rather than blindly echoing headers — distinguishing a real WebSocket server from a misconfigured cache, a transparent proxy, or an intermediary that would corrupt the subsequent framing. Cheap proof-of-understanding before the protocol leaves HTTP behind.

Q: Your autoscaler scales on CPU, but the WebSocket fleet’s memory grows steadily with no CPU change. What is happening? A: Connections accumulate state without generating traffic: each idle connection holds fd, buffers, and per-connection context, and new clients arrive faster than churn removes them — memory, not CPU, is the true capacity signal. Fixes: scale on connection count or memory, budget memory per connection explicitly, and enforce per-instance connection caps so one instance cannot silently overcommit. CPU-based autoscaling is structurally blind to idle-connection growth.

Q: Why do many WebSocket teams add an application-level ping despite TCP keepalive existing? A: They detect different failures: TCP keepalive detects dead peers after kernel-level idle periods (defaults measured in hours) and proves only that the TCP stack is alive — not the application, its event loop, or the intermediaries. An app-level ping inside every intermediary’s idle window prevents idle-timeout resets, detects hung application state a TCP ACK would mask, and gives per-connection latency. They compose: keepalive for the kernel’s view, ping for the application’s.

Q: A colleague proposes replacing the WebSocket fleet with SSE to cut complexity. What do you check before agreeing? A: Directionality first: if clients must send meaningful data (chat, collaborative editing, acks), SSE alone is insufficient and the client falls back to separate HTTP requests — acceptable for some workloads, latency-hostile for others. Then: proxy behavior (SSE is plain HTTP and friendlier to intermediaries), reconnect semantics (SSE’s built-in Last-Event-ID is a real win), and fan-out architecture (unchanged — SSE has the same per-connection state and fan-out problem). The transport simplification is real; the state problem is not.

Key Takeaways

• A WebSocket is an HTTP-borne upgrade into a different framing protocol: the 101 is a contract with intermediaries, and everything that fails afterward fails at an intermediary that never read it.
• Connection state lives in exactly one process on one machine — which makes sticky routing a correctness requirement, deploys a drain problem, and memory-per-connection the real capacity unit.
• Sticky sessions and pub/sub fan-out differ in where routing lives: per-connection pinning versus subscriptions-as-data in a broker; the fan-out arithmetic decides the architecture.
• Reconnect storms are synchronized transitions: jitter, admission-limited handshakes, and waved deploys turn a herd into a spread; capped backoff without jitter re-synchronizes every attempt.
• The transport choice follows the data flow — WebSocket for managed bidirectional push, SSE for one-way streams, polling for slow data, QUIC migration for mobile — not fashion.

Cross-References

• HTTP Protocol Evolution — the sibling chain owning HTTP/1.1 semantics, multiplexing, and the protocol landscape the upgrade tunnels through.
• TCP Lifecycle — the connection underneath: handshake, states, and the kernel-side keepalive distinction.
• Load Balancing — session affinity mechanics and the failover cliff sticky fleets inherit.
• Virtual Threads vs Async — the C10K connection model: how a runtime holds thousands of sockets cheaply.
• Message Queues — the delivery-semantics backbone for pub/sub fan-out and durable server-side queues.
• Kubernetes Pod Lifecycle — the SIGTERM/drain contract long-lived-connection servers must implement to survive orchestrators.
• Redis Caching — the stampede pattern in its original form, identical to the reconnect storm’s arithmetic.
• Backpressure — what a slow WebSocket client does to its server’s dispatch loop, and the flow-control answers.
• Webhook Delivery — the outbound-push sibling with delivery guarantees, retries, and the online/offline split.
• TCP Guarantees Delivery — why a reliable transport still leaves application-level delivery (and the offline path) unsolved.
• WebSocket Protocol — the book chapter on framing, masking, and close semantics.
• WebSocket — the practical client/server API and deployment notes.
• Pub/Sub — the pattern and its at-most-once pitfalls behind naive Redis fan-out.
• Kafka — the durable log that turns fan-out into a retryable, replayable pipeline.
• IO Multiplexing — the event-loop mechanics that make thousands of connections per instance affordable.
• QUIC Connection Migration — how transport-layer migration removes a class of mobile reconnects.
• RFC 6455 — the WebSocket protocol specification: handshake, framing, masking rationale (verified this session).