Pick a depth. Each prompt opens in your AI pre-loaded with the lesson. Click a row to preview the prompt.
CUDA kernel launches are asynchronous: the CPU issues the launch and immediately moves on to the next line of Python, while the GPU executes the kernel on its own timeline. Wrap a GPU op in plain time.time() calls without a synchronization point and you'll measure how fast the CPU can queue work, not how fast the GPU does it — the number will look absurdly fast and be completely wrong. The fix is either torch.cuda.synchronize() around the region you're timing, or better, CUDA Events, which are markers recorded directly on the GPU's own timeline and measured with essentially zero overhead. This is not academic: every profiling task for the rest of this course depends on timing GPU work correctly, and a wrong number here silently invalidates every optimization decision built on top of it.
The demo times the same matmul three ways — naive (wrong), with a manual synchronize (correct but coarse), and with CUDA Events (correct and precise) — so you see the naive number lie in real numbers, not just in theory.
start.record() and end.record().import torch, time
device = "cuda" if torch.cuda.is_available() else "cpu"
a = torch.randn(4096, 4096, device=device, dtype=torch.float16)
b = torch.randn(4096, 4096, device=device, dtype=torch.float16)
# 1. Naive: no sync. On GPU this measures launch overhead, not kernel time.
t0 = time.perf_counter()
for _ in range(20):
c = a @ b
naive_ms = (time.perf_counter() - t0) / 20 * 1000
print(f"naive (no sync): {naive_ms:.4f} ms <- likely a lie on GPU")
# 2. Correct but coarse: bracket with a single synchronize.
if torch.cuda.is_available():
torch.cuda.synchronize()
t0 = time.perf_counter()
for _ in range(20):
c = a @ b
if torch.cuda.is_available():
torch.cuda.synchronize()
sync_ms = (time.perf_counter() - t0) / 20 * 1000
print(f"synchronized: {sync_ms:.4f} ms")
# 3. Correct and precise: CUDA Events, recorded on the GPU's own timeline.
if torch.cuda.is_available():
start, end = torch.cuda.Event(enable_timing=True), torch.cuda.Event(enable_timing=True)
start.record()
for _ in range(20):
c = a @ b
end.record()
torch.cuda.synchronize()
print(f"cuda events: {start.elapsed_time(end) / 20:.4f} ms")python3 main.py