Every architecture diagram assumes the machine underneath is infinite. It is not. Cores, packets, handshakes, sockets and blast radius.
Most system design teaching stops at the box. Draw a load balancer, draw three app servers, draw a cache, and the diagram is done — as though each of those rectangles could absorb whatever you sent it. At a million requests a second the rectangles fight back. One core will do a few thousand TLS handshakes and no more. One network card will drop packets long before the bandwidth graph looks busy, and the counter that tells you is buried in a file most builders have never opened. One machine can open about twenty-eight thousand connections to a single destination, which is a number nobody discovers until production. This course goes under the diagram. You work out what a million requests a second is in cores, packets, bytes and money; find which resource on one machine actually binds first; follow a packet from the wire to your accept queue and learn where it gets dropped; cost TLS as CPU rather than latency; measure what HTTP/3 really charges you for its better loss behaviour; and meet the socket and port limits that only appear when a fleet talks to a fleet. Then it goes global — what anycast is under the cloud console, how big a cell should be, how to compute blast radius with combinatorics instead of hope, how to ship to thirty locations without an outage, and what standby capacity actually costs in rupees and dollars. Every ceiling comes with the arithmetic that predicts it and the command that proves it on your own machine.
Built by Lakshya Kumar
Paste this into any AI chat. Fill in the bracketed parts with your context — you'll get back a straight answer on whether this belongs on your plate.
We grant free access case-by-case — students, career-switchers, builders on a tight budget. Sign in to send us a note.
Sign in to applyFinished the tasks? Take the prompt to your AI and get tested on it. We copy the prompt and open the app — just paste it in.
Find the real limit of one machine, and which of cycles, memory, packets or syscalls binds first — that decides whether more boxes help.
Follow a packet from the wire to your read(), find the queue it died in, and change only the knob whose counter you can watch move.
Asymmetric crypto is billed per connection, symmetric per byte. Measure the split with openssl speed, then size a fleet against your number.
Six sockets per client or one, a lost packet that stalls everything or nothing, and the CPU bill for moving your transport into userspace.
TIME_WAIT, conntrack tables, descriptor caps and a shared NAT budget — accounting that stays invisible until many boxes call one endpoint.
The same address announced from many places, who decides which one a user reaches, and what a drain costs in reset connections.
Blast radius is the product, not scale. Size an independent copy of the stack, route to it, move a tenant between them, price the globals.
Give each tenant a random subset of the fleet and the reach of a bad neighbour stops being a hope and becomes a number you can compute.
Most outages follow a change, so the rollout is the control: exposure ladders, bake times you derive, and rollback across thirty sites.
Lose one of three regions and survivors inherit 1.5x. Derive the ceiling, price the idle fleet in rupees and dollars, then prove it holds.
Follow one request from the resolver to the wire, attach a number to every hop, and learn which ceiling binds first from 1k to 1M rps.
Complete all modules, then submit the required number of capstone projects. Each must earn a passing rating from an admin reviewer.
Take one real service — yours, or something you stand up for this — and find out what one box actually does with it. Load it until throughput stops rising, then prove which resource is binding: CPU, memory bandwidth, packets per second, or syscall overhead. The deliverable is not a number, it is the evidence: the load-generator command, the counters before and after, and the reasoning that rules out the other three candidates. Then raise the ceiling once and show the next thing that binds.
I am working through A Million Requests a Second on Capstok, which covers the layers underneath a system-design diagram: single-machine ceilings, the kernel packet path, TLS as CPU cost, HTTP/2 and QUIC throughput, socket and port limits, anycast and BGP, cell-based architecture, shuffle sharding, global deploy waves, and standby-capacity economics. Here is my context: 1. The service I work on and roughly its peak requests per second: [describe it] 2. What I currently believe its bottleneck is: [your honest guess] 3. What I have actually measured, if anything: [be honest, none is a valid answer] Given that, do four things: - Tell me which layer is most likely binding for my service, and why my guess is or is not plausible. - Give me the three commands I should run first to confirm or kill that theory, and tell me what output would mean I am wrong. - Do the arithmetic for my numbers: what my current traffic implies in packets per second, handshakes per second and cores. - Name the ceiling I will hit next if I raise the current one, so I know what I am walking into.
Produce a full capacity plan for a service of your choosing at one million requests a second at peak. Work every layer this course covered: connections and protocol choice, TLS handshake CPU, packets per second and instance PPS allowance, app fleet size derived from measured per-request cost, cell count and size against a stated acceptable-impact target, region count, and the headroom needed to survive losing one region. Give the monthly bill in both rupees and dollars, and name the three assumptions most likely to be wrong.
Pick a system you work on or know well and answer, with arithmetic rather than adjectives, what fraction of users a single failure takes down. Identify every genuinely global component — the router, the control plane, the config push, the shared database, the one flag service everything reads — because those are the real answer. Then propose one change that measurably reduces the worst case, and state its cost. A finding of the shape it is currently one hundred percent and here is why is a perfectly good result if you show the work.
Choose a limit this course describes — ephemeral port exhaustion, conntrack table full, accept-queue overflow, a softirq pinned to one core, TLS handshakes saturating CPU — and build the smallest harness that reproduces it reliably on a machine you control. Show the failure, show the counter that identifies it, apply the fix, and show the same harness passing. Include the part that matters most: what the failure looked like from the outside before you knew what it was.
Design the deployment plan for a change going to a global fleet. Derive the wave sizes from a stated risk budget rather than picking round numbers, derive bake time from the slowest failure mode you are actually screening for, define the automatic rollback signal and the comparison it is made against, and handle the part that cannot be rolled back at all. State the total wall-clock time and defend it. Then write the paragraph explaining to someone impatient why it is not faster.
The canonical write-up of shuffle sharding, and where the arithmetic in module 9 comes from.