Pick a depth. Each prompt opens in your AI pre-loaded with the lesson. Click a row to preview the prompt.
Almost every failed chat backend fails the same way: the process holding the WebSocket also does the work. It queries the database, renders the payload, checks permissions, and writes to the other sockets. It seems efficient — one hop, no serialisation — right up to the first deploy, when restarting that process disconnects three hundred thousand people simultaneously and they all reconnect inside two seconds. Stateful and stateless code have opposite operational needs. Socket-holding processes want to live for days and be drained slowly; business logic wants to be redeployed twelve times an afternoon and scaled on CPU. Putting them in one binary means you get the worst of both: you cannot deploy without a reconnect storm, and you cannot scale the CPU-hungry part without also moving connections. The separation is the first real decision in the design, and it comes with a second one that is even more consequential: the order in which a send is sequenced, stored, acknowledged and fanned out. Get that order wrong and you have a system that occasionally tells a user their message was sent when it no longer exists anywhere.
Trace one message end to end. The sequence diagram below is the whole architecture: four tiers, and a strict ordering of the steps between them.
The hard rule is visible in the diagram as the position of the ack. The message service asks the sequencer for a number, writes the immutable row, and only then tells the sender it worked. Fan-out happens after the ack, not before. That order costs you roughly one storage round trip of latency on the send path — around 10 to 30 milliseconds — and it buys the only durability guarantee that users actually notice.
The reverse order is tempting and is what a naive implementation does: relay to the other sockets first because it feels faster, persist afterwards. It works perfectly until the storage write fails. Now some recipients have a message on screen that no history read will ever return, and the sender was told it was fine. That is worse than an error, because nobody can tell it happened.
The second thing to notice is the session registry. Something has to answer the question “which gateway node currently holds a socket for user 4471, device B?” — and that answer changes thousands of times a second across the fleet. It is a small, extremely hot, entirely rebuildable key-value store: sessions vanish when a gateway dies, and a client reconnecting rebuilds its own entry. That means it can live in memory with weak durability, which is a rare luxury in a system where everything else must be durable.
// TIER 1 — the connection gateway.
// It holds sockets and a routing table. It has no database handle,
// no permission logic and no templates. Its entire job is
// (sessionID -> socket) plus a write that can never block fanout.
type Session struct {
ID string
UserID string
DeviceID string
out chan []byte // BOUNDED. A slow phone must not stall a channel.
conn *websocket.Conn
}
type Hub struct {
mu sync.RWMutex
sessions map[string]*Session // sessionID -> session
byUser map[string][]string // userID -> sessionIDs (multi-device)
// channel -> live sessionIDs, maintained incrementally on
// subscribe/unsubscribe. This is what makes fanout O(connected)
// instead of O(members). Challenge 5 lives or dies on it.
byChannel map[string]map[string]struct{}
}
// Deliver writes ONE pre-serialized frame to many sessions.
// The frame is encoded once by the caller: at 8,000 recipients,
// per-recipient JSON encoding is the entire CPU budget of the node.
func (h *Hub) Deliver(sessionIDs []string, frame []byte) (sent, dropped int) {
h.mu.RLock()
defer h.mu.RUnlock()
for _, id := range sessionIDs {
s, ok := h.sessions[id]
if !ok {
continue // reconnected elsewhere; registry is eventually consistent
}
select {
case s.out <- frame:
sent++
default:
// Outbox full: this client is not draining. Never block here —
// one bad mobile connection would otherwise stall the channel.
// Evict it. It reconnects and catches up by cursor (challenge 6),
// so dropping a frame is recoverable; blocking is not.
dropped++
go h.evict(id, "outbox_full")
}
}
return sent, dropped
}
// Draining for deploy: stop accepting new sessions, then close in
// batches with jitter so 50,000 clients do not reconnect in one second.
func (h *Hub) Drain(ctx context.Context, batch int, gap time.Duration) {
h.acceptNew.Store(false)
for _, id := range h.snapshotSessionIDs() {
h.closeWithReason(id, "server_draining_reconnect_soon")
if n := h.closedSinceGap.Add(1); int(n)%batch == 0 {
select {
case <-ctx.Done():
return
case <-time.After(gap):
}
}
}
}go run main.go