Two means: Welch, pooled and paired t tests
H₀: μ₁ = μ₂ · H₁: μ₁ ≠ μ₂
t = -17.3488 · df = 11986.1682 · p = 1.31E-66
p = 1.31E-66 ≤ α = 0.05: reject H₀. The gap between the means, -1.0898, is larger than chance alone would easily produce; the interval, [-1.2129, -0.9666], shows how large it could be.
How to read the picture
The curve is the null world: how t would vary if the two population means were equal.
The vertical line is your t: the gap between the means in standard errors.
The shaded tail is the p-value; the lighter region is where t must land to reject H₀ at α.
Below, the interval for the gap: when it excludes 0, the two-sided test rejects.
Your data
Test
Alternative hypothesis
Significance level α
Helion's customers on Portal v2 paid in 14.36 days on average, against 15.44 days on the old portal. Is the gap real? Choose Welch's test (the default), the pooled test, or the paired test for the same units measured twice, and move the numbers: the t statistic lands on the curve of the null world, and the interval shows how large the gap could be.
The estimate and what the data show
The worked steps
Difference in means
x̄₁ − x̄₂ = 14.3551 − 15.4449 = -1.0898Each group's variance of the mean
s₁²/n₁ = 3.4039² / 6031 = 0.001921 · s₂²/n₂ = 3.4764² / 5969 = 0.002025Standard error of the difference
SE = √(s₁²/n₁ + s₂²/n₂) = √(0.001921 + 0.002025) = 0.06282The t statistic
t = (x̄₁ − x̄₂) / SE = -1.0898 / 0.06282 = -17.3488Welch's degrees of freedom
df = (0.001921 + 0.002025)² / (0.001921² / 6030 + 0.002025² / 5968) = 11986.1682The p-value
p = 2 × P(T ≥ |-17.3488|) = 1.31E-66Critical value at α
α = 0.05: t* = ±1.9602Confidence interval for the difference
95 %: x̄₁ − x̄₂ ∓ t* × SE = [-1.2129, -0.9666]Effect size (Cohen's d)
d = (x̄₁ − x̄₂) / s_p = -0.3168
The same in code
- scipy, from the summary statistics
from scipy import stats stats.ttest_ind_from_stats(14.355122, 3.403854, 6031, 15.444899, 3.476403, 5969, equal_var=False, alternative='two-sided')- scipy, from the data
from scipy import stats stats.ttest_ind(a, b, equal_var=False, alternative='two-sided')- statsmodels, from the data
from statsmodels.stats.weightstats import CompareMeans, DescrStatsW CompareMeans(DescrStatsW(a), DescrStatsW(b)).ttest_ind(usevar='unequal', alternative='two-sided')- Excel
=T.TEST(A2:A6032, B2:B5970, 2, 3) B1: t = -17.3488 B2: df = 11986.1682 B3: =T.DIST.2T(ABS(B1), B2)
Formulas
Welch's test keeps each group's own variance in the standard error, and its degrees of freedom shrink when the spreads or the sizes differ.
Try this
Pooled against Welch
Open the small noisy group against the large calm one, and switch between Welch's test and the pooled test.
Look for: The pooled test borrows the calm group's small spread for both, so its standard error is too small and its p-value too low. Welch's test keeps each group's own spread and fewer degrees of freedom.
Pairing removes the noise
Open the agents' paired test and drag r from 0.85 down to 0.
Look for: As r falls, the differences vary more, the standard error grows and t shrinks. The agents who are busy before are busy after: pairing takes that out of the comparison.
More agents, same gap
Open the first 12 agents and drag the number of pairs from 12 up to 80.
Look for: The gap and the spread stay, but the standard error shrinks with √n, so the interval narrows around the same mean difference.