Pick a depth. Each prompt opens in your AI pre-loaded with the lesson. Click a row to preview the prompt.
This is the idea the rest of the course is built on, and it is what separates people who can design systems from people who can only list them. When traffic goes up 100x, almost nothing about the logic changes. Token-bucket rate limiting is the same three lines of arithmetic at twelve thousand requests per second and at 1.2 million. Base62 encoding of a 64-bit identifier is the same function whether you have shortened a thousand URLs or a billion. Double-entry bookkeeping still requires that debits equal credits whether you settle ten payments a day or ten million. What changes is where the counter lives, who hands out the identifiers, and how the ledger is partitioned — deployment concerns wearing an algorithms costume. Builders who cannot see that seam rewrite working logic under pressure and ship correctness bugs into the one part of the system that was fine. Builders who can see it move state around and leave the correct code alone. Every stress task in this course ends with an explicit what-does-not-change paragraph for exactly this reason: naming the invariant is how you protect it.
Sort every line of a design into three layers, and the whole subject gets quieter.
Layer 1 — the contract. What must be true for the system to be correct. At most N requests per key per window. Debits equal credits. One driver is assigned to at most one active trip. A short code maps to exactly one URL, forever. This layer changes when the product changes, and essentially never because of load.
Layer 2 — the algorithm. How you compute the thing the contract requires. Token bucket. Base62 over a monotonic counter. Consistent hashing. A ranking function. This layer changes when you find a better method, which is rare, and it is almost never the right response to a traffic increase.
Layer 3 — the deployment. Where the state lives, how many copies exist, who coordinates them, what happens when one dies. In-process dictionary, or Redis, or sharded Redis with a local pre-check. Single primary, or primary plus replicas, or multi-region. This layer changes constantly, and it is where essentially all scale pressure lands.
The two code samples below implement the same rate limiter. The first runs in one process; the second runs across fifty servers behind a load balancer. Diff them and find the arithmetic. It is character-for-character identical apart from min becoming math.min. Every single other difference — the hash, the Lua script, the EXPIRE, the round trip — is layer 3.
That is the whole lesson, and here is the rule it produces: when you find yourself changing layer 1 under load, you are not scaling the system, you are changing the product. Relaxing a limit, dropping a uniqueness guarantee, allowing a double charge because deduplication got expensive — these are sometimes correct decisions, but they are product decisions and they need to be named as such rather than smuggled in as an optimisation. In an interview, saying I would move the counter into Redis and shard by key, but the bucket maths is unchanged and the guarantee still holds is worth more than any component you could name, because it proves you know which part was load-bearing.
The reverse failure is just as common and less obvious: people leave layer 3 alone and try to solve a deployment problem inside the algorithm. Adding a cleverer eviction policy will not save a cache whose real problem is that it lives in one region and half your users are on another continent.
# Single process. 1 server, ~12,000 requests/sec, state in a dict.
import time
class TokenBucket:
def __init__(self, capacity, refill_per_sec):
self.capacity = capacity
self.rate = refill_per_sec
self.tokens = capacity
self.updated = time.monotonic()
def allow(self, cost=1):
now = time.monotonic()
# ---- THE ALGORITHM (layer 2) -------------------------------
self.tokens = min(self.capacity,
self.tokens + (now - self.updated) * self.rate)
self.updated = now
if self.tokens >= cost:
self.tokens -= cost
return True
return False
# ------------------------------------------------------------
buckets = {} # <-- layer 3: the state lives here, in RAM, on one box
def allow(key, cap=100, rate=10.0):
if key not in buckets:
buckets[key] = TokenBucket(cap, rate)
return buckets[key].allow()
print([allow("user:42") for _ in range(105)].count(False)) # 5 rejectedpython3 main.py