How Much Traffic You Need Before Testing (Statistical Power)

Why this matters

Running conversion tests on a page with 200 monthly visitors is cargo-cult CRO — you will reach any conclusion you want before the result means anything, and you will ship changes based on noise. Understanding minimum traffic thresholds before running tests is what separates a learning program from a superstition factory.

Try it yourself

1. Use a sample size calculator to determine the minimum monthly unique visitors your highest-priority page needs before a 20% relative lift test is feasible in under 30 days.
2. Check whether any page currently being tested by your team is below that threshold — if so, identify what qualitative method (session recordings, surveys) should replace the A/B test.
3. Calculate the statistical power of your last A/B test by entering its actual sample size into a power analysis tool — note whether the test was underpowered.

References

• Evan Miller — Sample Size Calculator