Quiz me — Why Evals Are the Bottleneck

quiz

easy

7 questions · need 5/7 to pass.

Q1.What is the core claim of the 'eval-first' mindset?

single

Q2.Two classifiers are both 90% accurate. Which one should you prefer?

single

Q3.Which of these must you pin to make an eval score reproducible across runs?

multi-select

Q4.The four parts that every evaluation decomposes into are dataset, system-under-test, scorer, and ______.

fill in

Q5.Why is 'it passed my 3 test prompts' insufficient evidence a feature is reliable?

single

Q6.Select every statement that is true about offline vs online evaluation.

multi-select

Q7.Order the scorer types from cheapest/first-choice to most expensive/last-resort.

rank

Re-order from top (rank 1) to bottom.

1.Schema validation

2.Human review

3.Exact / regex match

4.Semantic similarity

5.LLM-as-judge