Project · Compute and verify a non-obvious probability

project

hard

module project

Ship something real. Submit your work when you're done.

Brief

Pick a real-world probabilistic situation where the intuitive answer is wrong, model it from first principles using only the axioms, derive the analytical answer step by step, and then verify it with a Monte Carlo simulation in Python. Your write-up should make the trap visible: name the intuitive wrong answer, then show why the formalism corrects it.

Deliverables

A 1–2 page write-up (PDF or Markdown) stating the problem, defining the sample space and events explicitly, and naming the 'intuitive wrong answer' that the problem is famous for inducing.
An analytic derivation of the probability — every step justified by an axiom, the multiplication rule, total probability, Bayes, or complementary counting (label which one at each step).
A Python script (or notebook) that simulates the situation and estimates the probability with $N \geq 100{,}000$ trials, using a fixed `random.seed` so the result is reproducible.
A side-by-side numerical comparison of the analytic answer and the Monte Carlo estimate, with one sentence explaining why the discrepancy is of expected magnitude (roughly $1/\sqrt{N}$).

How we grade it

The sample space is written down before any formula is applied — no implicit equally-likely assumptions, and the choice of ordered-vs-unordered outcomes is justified.
The analytic derivation cites which axiom or theorem (multiplication rule, total probability, Bayes, complementary counting) is used at each step.
The Monte Carlo simulation matches the analytic answer to within roughly $3/\sqrt{N}$ — i.e., within three standard errors.
The write-up names at least one concrete way an over-eager reader could get the wrong answer (the 'trap'), and explains how the chosen formalism avoids it.
Code is reproducible: a single `python script.py` produces the printed estimate and the seed is fixed.

Hints

Good problem candidates: the Monty Hall problem, the boy-or-girl paradox, the two-envelope paradox, the gambler's ruin for short walks, false-positive rates on a rare disease, hash-collision probability on small bit-widths, the secretary problem, the German tank problem.

If you pick Monty Hall, write the sample space as triples

(\text{prize-door}, \text{chosen-door}, \text{opened-door})

— not as 'switch vs stay.' The mistake everyone makes lives in the sample space, not in the algebra.

For the medical-test variant, fix a prior, sensitivity, and specificity at the top of your script as named constants. Do not bury them in the formula — Bayes problems become unreadable when the inputs are anonymous.

N = 100{,}000

trials gives a standard error of roughly

1/\sqrt{N} \approx 0.003

on a probability estimate. Quote your simulation answer with that uncertainty in mind, not to 6 decimal places.

Always set random.seed(0) (or numpy equivalent) at the top so a grader can reproduce your number exactly.

Expected output

A document opening with something like:

Problem: In Monty Hall, after the host opens a goat door, should I switch? Sample space: triples (prize, choice, opened); 3 prize positions x 3 choices x (constrained) host openings. Assumptions: prize uniform over doors; contestant chooses uniformly; host always opens a goat door and picks uniformly when two are available. Naive intuition: 50/50 either way. Analytic answer: P(win | switch) = 2/3. Simulation (N=200,000, seed=0): 0.66712 (expected SE ~ 0.0011).

…followed by the worked derivation citing axioms / rules at each step, and the simulation script.

Stretch goals

Vary an assumption (e.g. host opens randomly, possibly revealing the prize) and recompute both analytically and via simulation. Does switching still help, and by how much?

Generalise to

n

doors and

k

host-opened goat doors. Is the 'switch' advantage shrinking, growing, or invariant?

Solve the same problem two ways: once via the sample-space / counting approach, and once via Bayes' rule with explicit prior and likelihood. Which derivation is shorter? Which one makes the assumptions more visible to a reader?

Plot the running Monte Carlo estimate as a function of trial count

n

on a log-x axis, with a

\pm 1/\sqrt{n}

envelope around the analytic answer. Confirm the estimate stays inside the envelope as grows.