A/B Test Significance Calculator
Enter the visitors and conversions for your control and your variant to see whether the difference is statistically significant, how big the lift is, and how confident you can be in it.
Your test results
Result
Enter visitors and conversions for both versions.
Sample size calculator
Before you start, work out how many visitors each version needs so the test can detect the lift you care about.
Relative: 15% takes 3% to 3.45%.
Visitors needed
Enter your current conversion rate and the lift you want to detect.
Frequently Asked Questions
What does statistically significant mean in an A/B test?⌄
It means the difference between the two versions is larger than you would expect from random variation alone, at the confidence level you chose. At 95% confidence, a difference this size would show up by chance less than 5% of the time if the two versions actually performed the same.
Which test does this calculator use?⌄
A two-sided two-proportion z-test with a pooled standard error, the standard method for comparing conversion rates. Two-sided means it checks for a difference in either direction, so a variant that does worse is caught too. The confidence interval for the difference uses the unpooled standard error.
Can I stop the test as soon as it turns significant?⌄
Better not. Checking repeatedly and stopping at the first significant reading inflates the false-positive rate well above the level you picked. Decide the sample size before the test starts (the sample size calculator on this page does that), run the test until you reach it, and ideally cover at least one full week so weekday and weekend behavior are both included.
Why does a small lift need so many visitors?⌄
The smaller the difference you want to detect, the more data it takes to tell it apart from noise. Halving the lift you want to detect roughly quadruples the visitors you need, and low baseline conversion rates push the number up further.
Is my data sent anywhere?⌄
No. The calculations run in your browser and nothing you type is sent to our servers.