2 Sample Z Test Calculator

What a two sample z test answers

A two sample z test asks whether two group averages differ by more than random noise allows. It takes a mean, a standard deviation and a sample size from each group, measures the gap between the two means, and weighs that gap against how much it would wobble if the study were repeated. The result is one number, the z statistic, and a p value that states how often chance alone would open a gap this wide if the two populations do share the same mean.

The test weighs evidence against one specific claim, the null hypothesis that both populations have the same mean. A small p value says the observed gap would be rare under that claim, so you reject it. A large p value says the gap fits comfortably inside everyday sampling noise, so you keep it. Neither outcome proves a cause, and neither says the difference is large. It only says how surprised you should be.

The formula, and the two variance choices

The calculator uses the unpooled form of the test statistic:

z = (x̄1 − x̄2) / sqrt(s1²/n1 + s2²/n2)

x̄1 and x̄2 are the two sample means, s1 and s2 the two standard deviations, n1 and n2 the two sample sizes. Each group contributes its own squared spread over its own size, the two ratios are added, and the square root turns the sum into the standard error of the difference between the means. The z statistic is the observed gap counted in standard errors.

A pooled variant replaces the two separate variances with one combined variance:

s_p² = ((n1−1)s1² + (n2−1)s2²) / (n1+n2−2), then z = (x̄1 − x̄2) / sqrt(s_p² (1/n1 + 1/n2))

The pooled version assumes both populations share the same variance, and it buys a small gain in precision when that assumption holds. Use it when the two standard deviations are close, for example within about 20 percent of each other, and both samples are of similar size. When the spreads differ, the unpooled form above is the safer choice, which is why the calculator leads with it. Section 6 runs both versions on the same data so you can see how little they disagree when the assumption is reasonable.

When z is valid and when you need t

The z test treats the standard deviations as known numbers rather than estimates. That is the right model when a production process has a documented long run standard deviation, or when theory fixes the spread. It is also the accepted approximation when both samples are large, because a sample standard deviation from 30 or more observations is stable enough that the normal distribution describes the test statistic well. Penn State's STAT 500 course notes present exactly this split: known population standard deviations or large samples point to z, small samples with estimated spreads point to t.

The t test exists because an estimated standard deviation adds uncertainty, and with few observations that uncertainty is real. With n below 30 in a group, and the standard deviation estimated from that same small sample, the t distribution is the honest tool. The two distributions converge as the sample sizes grow, so the decision between them matters less once n passes a few dozen per group.

The test also assumes the two samples are independent, meaning no unit appears in both groups, and that each population is roughly normal, or the sample large enough for averages to sit near normal. Standard deviations that are wildly unequal with small samples also break the test, and no choice of z or t repairs that on its own.

Worked example: two teaching methods

Say two groups of students take the same final exam. Method A averaged 82 with a standard deviation of 6 across 40 students. Method B averaged 79 with a standard deviation of 5 across 45 students. The question is whether a 3 point gap is noise or signal. Set a two tailed test at the 0.05 level and compute step by step.

Step 6 uses the standard normal cumulative distribution, the function tabulated in the NIST handbook shown in section 7. The p value of 0.0129 sits below 0.05, so the test rejects the null hypothesis of equal means. A 95 percent confidence interval lands on the same call: 3 ± 1.960 × 1.2065 runs from 0.64 to 5.36, and the interval misses zero. The gap is between roughly two and three standard errors wide depending on the variance choice, which is why the p value is small without being extreme.

One tailed against two tailed: a delivery example

A courier wants to know whether a new routing rule raised average delivery time, not just changed it, so a one tailed test fits. The old route logged 150 deliveries with a mean of 32.4 minutes and a standard deviation of 8.1. The new route logged 175 deliveries with a mean of 30.9 and a standard deviation of 7.6. The gap is 32.4 − 30.9 = 1.5 minutes, and the standard error is sqrt(8.1²/150 + 7.6²/175) = sqrt(0.437 + 0.330) = sqrt(0.767) = 0.876. The z statistic is 1.5 / 0.876 = 1.712.

The two rows answer different questions, which is why they disagree. The one tailed p value is exactly half the two tailed value, because it puts the whole 0.05 budget in one tail. Pick the tail before seeing the data. Choosing one tail after a first look, because it happens to be significant, is the classic way to manufacture a result that will not replicate.

The pooled version on the same data

Return to the teaching example and pool. The combined variance is s_p² = (39 × 36 + 44 × 25) / (40 + 45 − 2) = (1,404 + 1,100) / 83 = 2,504 / 83 = 30.169. The standard error becomes sqrt(30.169 × (1/40 + 1/45)) = sqrt(30.169 × 0.0472) = sqrt(1.425) = 1.194, and the pooled z statistic is 3 / 1.194 = 2.513, with a two tailed p value of 0.0120.

The two calls agree here because the sample standard deviations, 6 and 5, are close and both samples are healthy sized. The pooled version borrows strength from the assumption of a common variance, so it lands slightly smaller standard error, slightly larger z, slightly smaller p. With standard deviations of 3 and 12 on small samples the gap between the two versions would widen, and the unpooled form would be the one to trust.

Critical values and the decision rule

An older but still common way to run the test skips p values. You compare the z statistic against a fixed critical value that marks the edge of the rejection region. The values below come from the standard normal table in the NIST/SEMATECH e-Handbook of Statistical Methods (NISTIR 7787, 2012), whose cumulative distribution table is the same function the calculator uses for p values.

Reject the null hypothesis when the z statistic passes the critical value in the relevant direction: outside ±1.960 for a two tailed test at 0.05, above 1.645 for a one tailed test at 0.05, and so on. The teaching example's z of 2.487 clears 1.960, matching the p value call of 0.0129 below 0.05. The delivery example's z of 1.712 clears 1.645 in one tail but not 1.960 in two, which reproduces the table in section 5. Critical values and p values are two readings of the same curve, so they never disagree when applied to the same test.

Limits, mistakes, and what this test cannot do

Small samples with estimated spreads. If a group has fewer than about 30 observations and you estimated its standard deviation from those observations, a two sample t test is the honest choice. The z form will report a smaller p value than the data support.

Dependent samples. If the same people appear in both groups, before and after measurements for instance, the groups are not independent and the two sample test answers the wrong question. Use a paired test on the differences.

Percentages and conversion rates. Proportions need their own z test, built on counts rather than means and standard deviations. Entering a rate of 0.42 as a mean with a made up spread will not reproduce the correct result.

Extreme spread imbalance. One group with a standard deviation several times the other, combined with small samples, can distort both versions of the test. Larger samples soften this, or use a test built for unequal variances.

Significance is not size. A z of 2.487 on samples of 40 and 45 rejects the null, but the estimated gap is still just 3 points with a confidence interval of 0.64 to 5.36. Report the interval next to the decision so readers can judge whether the difference matters in practice, not only whether it clears a threshold.