Enter the visitors and conversions for versions A and B in this A/B test calculator. The calculator gives the conversion rates, relative uplift, p-value and a confidence interval for the difference, then tells you in plain words whether the result is significant. A sample size estimator shows how many visitors you need before you start. All calculations run in your browser.
Loading the tool… It runs in your browser and needs JavaScript.
Table of Contents
How to use the calculator
Type the number of visitors and conversions for the control (A) and the variant (B), and choose a confidence level: 90%, 95% (the usual choice) or 99%. The result updates as you type. For planning, enter your current conversion rate and the smallest improvement you care about in the sample size estimator, then add your daily traffic to see roughly how many days the test needs.
What the numbers mean
- Conversion rate: conversions divided by visitors for each version.
- Relative uplift: how much higher (or lower) B is than A as a share of A. Going from 5.0% to 5.7% is +14% relative, or +0.7 percentage points.
- p-value: the chance of seeing a difference at least this large if A and B actually performed the same. A p-value below 0.05 is significant at 95% confidence.
- Confidence interval: the range of plausible values for the true difference in percentage points. If it includes zero, the data are consistent with no difference.
How the test is calculated
The calculator runs a two-sided, two-proportion z-test. It pools both groups to estimate a common rate p, then computes z = (pB − pA) / √(p(1 − p)(1/nA + 1/nB)). The two-sided p-value comes from the normal distribution. This is the large-sample normal approximation described in the NIST/SEMATECH e-Handbook, and it gives the same p-value as a chi-square test without continuity correction.
The confidence interval uses the unpooled standard error. In borderline cases the interval and the p-value can disagree slightly. If fewer than 10 conversions or non-conversions are expected in any group, the approximation is rough, and the tool warns you.
Sample size
The estimator gives the visitors needed per variant to detect a change from the baseline rate to the baseline plus your minimum detectable effect. It uses the chosen confidence level and a power of 80%, 90% or 95%. Small effects on low baseline rates need a lot of traffic. Detecting a 10% relative lift on a 5% conversion rate at 95% confidence and 80% power needs about 31,000 visitors per variant.
Common A/B testing mistakes
- Stopping as soon as it turns significant. Checking repeatedly and stopping at the first p < 0.05 inflates false positives. Fix the sample size first.
- Running for less than a full week. Weekday and weekend visitors behave differently.
- Testing many variants or metrics at once. Each extra comparison raises the chance of a fluke winner.
- Ignoring practical size. A significant +0.1% lift may not be worth shipping.
A/B Test Calculator at a glance
Related guides: R Cheat Sheet: Vectors, Data Frames, Strings and Files · Excel Cheat Sheet: Formulas, Dynamic Arrays and Shortcuts · Python Cheat Sheet: Syntax, Data Types and File Handling.
Official documentation (checked 9 October 2026): NIST/SEMATECH e-Handbook of Statistical Methods: comparing two proportions · scipy.stats.chi2_contingency – SciPy Manual
Frequently asked questions
What confidence level should I use?
95% is the common default. Use 99% when a wrong decision is costly, and 90% for low-risk changes where speed matters more.
Why is my result not significant even though B looks better?
With small samples, random variation alone can produce differences of that size. Keep the test running until it reaches the planned sample size.
Is this a one-sided or two-sided test?
Two-sided. It asks whether B differs from A in either direction, which is the safer standard choice.