Quiz me — Why Evals Are the Bottleneck

quiz

easy

7 questions · need 5/7 to pass.

Q1.Why is 'it passed my 3 test prompts' insufficient evidence a feature is reliable?

single

Q2.The four parts that every evaluation decomposes into are dataset, system-under-test, scorer, and ______.

fill in

Q3.Two classifiers are both 90% accurate. Which one should you prefer?

single

Q4.What is the core claim of the 'eval-first' mindset?

single

Q5.Select every statement that is true about offline vs online evaluation.

multi-select

Q6.Which of these must you pin to make an eval score reproducible across runs?

multi-select

Q7.Order the scorer types from cheapest/first-choice to most expensive/last-resort.

rank

Re-order from top (rank 1) to bottom.

1.Semantic similarity

2.Human review

3.LLM-as-judge

4.Schema validation

5.Exact / regex match