- / A/B testing statistical significance calculator
A/B testing statistical significance calculator
Calculate statistical significance for your A/B tests. Enter audience and conversion data to determine if your results are statistically significant with confidence levels and p-values.
Campaign type
Variant A
Variant B
Confidence
The probability that the observed difference is not due to random chance. Higher confidence means more certainty but requires more data.
Hypothesis
One-sided tests if B is better than A. Two-sided tests if there's any difference between A and B.
Enter performance to check statistical significance
Variant A conversion rate
-
Variant B conversion rate
-
Power
The probability of detecting a true effect. Higher power (>80%) means the test is more likely to find a real difference if one exists.
-
p value
The probability of seeing this result if there was no real difference. Lower p-values (<0.05) indicate stronger evidence against the null hypothesis.
-
How to use this calculator
- Select the campaign type that matches your data. Labels for each variant update (for example, impressions for ads, visitors for web).
- Enter the exposure count and conversions for each variant of your A/B test.
- Select your desired confidence level (95% is standard for most tests).
- Choose one-sided if you only care whether B beats A, or two-sided if you want to detect any difference.
- Review the results to see if your test has reached statistical significance.
Note: Statistical significance indicates that the observed difference is unlikely due to chance. However, also consider practical significance (is the lift meaningful for your business?) before making decisions.
Browse more tools
Calendar link generator
Generate calendar event links for Google Calendar, Apple Calendar, Outlook, and more.
Code snippet generator
Transform your code into beautifully styled, email-compatible HTML snippets with syntax highlighting.
Mailto link generator
Create mailto links with pre-filled recipients, subject lines, body text, CC, and BCC fields.
