Project · Rate Your Own Box: Napkin First, Then Measure

project

hard

module project

Ship something real. Submit your work when you're done.

Brief

Pick one real HTTP endpoint you can run and load — your own service, or an open-source one you deploy locally. Before you measure anything, predict on paper what a single box will do: CPU microseconds per request, packets per request, the CPU, packets-per-second and bandwidth ceilings each of those implies, which one you expect to bind, and the fleet size and monthly bill for a million requests a second at peak. Then measure the same box and report the ratio between every prediction and every measurement. The deliverable that matters is a derate factor for your own stack, with the evidence that produced it.

Deliverables

A one-page napkin sheet, written and timestamped BEFORE any measurement, giving predicted CPU microseconds per request, packets per request, all three ceilings, and which one you expect to bind.
A capacity script in any language that takes rps, CPU microseconds per request, packets per request, response bytes, core count, utilisation target and per-GB egress price, and prints fleet size plus monthly compute and egress cost in both INR and USD.
Raw measurement output, saved as files: three `wrk2 -R --latency` runs at three distinct rates, `mpstat -P ALL 1 30`, `sar -n DEV 1 30`, `perf stat -p PID -e cycles -- sleep 30`, and `nstat -az` captured before and after each run.
A prediction-versus-measurement table with one row per quantity — CPU microseconds per request, packets per request, maximum sustained rps, p99 at half the ceiling — and a percentage-error column for each.
A short written verdict: your stack's derate factor, which ceiling actually bound, and one sentence naming the counter that proved it.

How we grade it

The napkin sheet's file timestamp is earlier than the earliest measurement file, and it states the expected binding ceiling explicitly as one of CPU, packets per second, or bandwidth.
Measured CPU per request is derived, not estimated: the submission shows both the `perf stat` cycle count and the request count for the same window, with the division and the clock speed used.
Load is generated in constant-arrival-rate mode at three distinct `-R` values with p50, p99 and p99.9 reported for each; a closed-loop `wrk` run on its own does not satisfy this item.
Every row of the prediction-versus-measurement table carries a numeric percentage error, and the stated derate factor equals measured maximum sustained rps divided by the predicted CPU ceiling.
Cost appears in both INR and USD with the conversion ratio stated, and compute and egress are separate lines with their ratio computed.

Hints

Run the load generator on a different machine from the service, or you will be measuring the two competing for the same cores. Pin the service to a known core set with taskset so perf stat -p PID and mpstat -P ALL tell a consistent story. Use keep-alive in the load generator unless you are deliberately measuring the handshake ceiling — mixing the two produces a number that describes neither. Find the maximum sustained rate by stepping -R upward until p99 starts climbing and stays climbing, not by watching for errors; the tail degrades well before anything returns a 500. Capture nstat -az immediately before and immediately after each run and diff them, because these counters are cumulative since boot and the delta is the only useful part. If %soft is concentrated on one core, check ethtool -l before you conclude anything about CPU cost. And be honest in the napkin sheet: a prediction you quietly adjust after seeing the measurement teaches you nothing.

Expected output

napkin.md written 14:02, measurements begin 14:31 PREDICTED cpu 95 us/req | 4.5 pkts/req CPU ceiling 202,105 rps pps ceiling 1,777,778 rps bandwidth ceiling 1,525,879 rps expected to bind: CPU MEASURED (16-core box, 3.0 GHz sustained, 25 Gbps) perf stat: 1,412,000,000,000 cycles over 30 s / 611,000 requests = 2,311,000 cycles/req = 770 us wall-equivalent, 141 us CPU max sustained (p99 stable): 61,400 rps at -R 60000 p50 2.1 ms | p99 18.4 ms | p99.9 47.2 ms mpstat: %usr 71 avg, %soft 9 avg (evenly spread, RSS on) nstat delta: ListenOverflows 0, TCPSynRetrans 3 PREDICTION vs MEASUREMENT cpu us/req 95 predicted -> 141 measured (+48%) packets/req 4.5 predicted -> 4.7 measured (+4%) max sustained 202,105 predicted -> 61,400 measured (-70%) DERATE = 61,400 / 202,105 = 0.30 BOUND BY: CPU (%usr 71, softirq spread, no listen overflow) FLEET for 1,000,000 rps at peak, 1.5x headroom: 25 boxes compute $24,820/mo Rs 16,50,530/mo (at Rs 66.5/$) egress 2.12 PB/mo $180,486/mo Rs 1,20,02,329/mo egress is 7.3x compute

Stretch goals

Repeat the whole exercise with keep-alive disabled in the load generator and report the new ceiling. The gap between the two is your handshake tax, in requests per second and in machines.

Disable multi-queue receive with ethtool -L eth0 combined 1, re-run, and record how far the ceiling falls and which mpstat column moved. Restore it afterwards.

Build the diurnal shape from a week of your own traffic, compute peak-to-mean, and re-derive the fleet from your measured per-box rating rather than the example numbers.

Halve your response payload by removing fields, re-measure, and report the change in packets per request, bandwidth ceiling and monthly egress in both currencies.

Take one published throughput claim from an engineering blog post, run it through the four smell tests, and write 200 words on what the number most likely counts and across how much hardware.