Every Project Ships One Measured Number

Why this matters

The difference between a portfolio project and a tutorial is not the tech stack, it is whether anything was measured. Accuracy delta against a baseline, availability through an induced outage, agreement with a human labeler, break-even request volume — one number, honestly obtained, does more work in an interview than three more repositories. The reason is that a number forces every earlier decision to be real: you cannot report accuracy by hop count without building a stratified question set, and you cannot report break-even without counting idle GPU time. Pick the number first and the rest of the project stops being optional.

Demo

Write the claim you intend to make before you start, as a sentence with a blank in it. If you cannot fill the blank with a measurement procedure, the project is not scoped yet. Keep this file in the repo root and update it as the real numbers land — the gap between the predicted and actual number is itself interesting.

Try it yourself

1. Write claim.md for the project you intend to build, with the blanks still blank.
2. Name the baseline you will compare against. If there is no baseline, the number means nothing.
3. Write the falsification line — the result that would mean the project failed. Keep it in the repo.
4. Estimate the honest cost (latency, dollars, build time) you will report alongside the win.

References

• Hamel Husain — Your AI product needs evals

Chat about this lesson