Project · Profile a model you already have and name its bottleneck

Take any model you can already run — a Hugging Face model, your LLM-from-scratch project, or a public repo — and produce a written bottleneck report for one forward+backward step: a correctly-timed wall-clock breakdown (CUDA Events, not naive time.time()), an arithmetic-intensity estimate for its biggest matmul, a full memory-budget breakdown (params/gradients/optimizer/activations), and a one-paragraph verdict naming whether the step is compute-, memory-, or communication-bound and why.

• A CUDA-Event-timed breakdown of one forward+backward step (or one inference step if you don't have gradients to compute).
• An arithmetic-intensity calculation, by hand, for the model's single largest matmul shape, classified against a ridge point you compute for your GPU.
• A full memory-budget table (params, gradients, optimizer state, activations) for your actual training/inference config.
• A one-paragraph verdict naming the dominant bottleneck (compute, memory, or communication) with the specific number that justifies it.

• Timing numbers are collected with CUDA Events or a verified `torch.cuda.synchronize()`, not naive wall-clock timing.
• The arithmetic-intensity calculation shows its FLOPs and bytes formulas explicitly, not just a final number.
• The memory-budget table accounts for all four categories and the total is checked against actual observed GPU memory usage.
• The verdict names one specific bottleneck and states the evidence, not a vague 'it seems compute heavy.'