One-way ANOVA and after

What you just saw

Helion Energy wants to know whether days to payment differ between its four regions, North, South, East and West. The animation took 40 customers at random from each region and asked one question: are the four averages further apart than luck alone would put them?

Each dot's distance from the grand mean was split in two: a within part, from the dot to its own region's mean, and a between part, from its region's mean to the grand mean. Squared and added up, the within parts measure how much customers vary inside a region; the between parts measure how far the regions sit apart. Their ratio, adjusted for how many pieces went into each, is F. For these 160 customers F = 6.69 on 3 and 156 degrees of freedom, far out in the F curve's tail: p = 0.0003. Even 40 customers a region are enough to see that the regions differ.

Why not just run all the pairwise t tests?

With four groups there are six pairs. Each t test at α = 0.05 has a 5 % false-positive rate, and those risks accumulate. Treating the six tests as independent (roughly right), the chance of at least one false positive is:

A 26 % chance of a spurious "significant" pair. With ten groups there are 45 pairs, and \(1 - 0.95^{45} = 0.901\): it exceeds 90 %. This is the multiple comparisons problem: the family-wise error rate, the chance of at least one false alarm across the whole family of tests, grows with every test you add. It is the reason ANOVA exists: it asks the question once, at 5 %.

How ANOVA works

ANOVA stands for analysis of variance, which is a misleading name: it compares means by analysing variances. The hypotheses are about the four regions' true mean days to payment:

If all groups share a mean, the spread between the group means is just sampling noise, so it should be about the same size as the spread within the groups, and F is close to 1. When the groups really differ, the between-group variance inflates and F grows. Written out:

In words: \(k\) is the number of groups (4 regions), \(N\) the number of customers in all (12,000), \(n_i\) and \(\bar{x}_i\) each region's size and mean, and \(\bar{x}\) the grand mean. The top is the between mean square: how far the region means sit from the grand mean, weighted by region size, per degree of freedom. The bottom is the within mean square: how far customers sit from their own region's mean, per degree of freedom; it is the pooled variance of the four regions.

For all 12,000 Helion customers the region means are 14.09 days (North, 3,600 customers), 14.98 (South, 3,413), 16.44 (East, 2,622) and 14.29 (West, 2,365). Step by step:

• The between sum of squares is 9,468 on k − 1 = 3 degrees of freedom: a between mean square of 9,468 ÷ 3 = 3,156.
• The within sum of squares is 136,085 on N − k = 11,996 degrees of freedom: a within mean square of 11.34, the pooled variance of days to payment inside a region.
• F = 3,156 ÷ 11.34 = 278.2. If the regions did not differ, F would be close to 1; the chance of an F this large on (3, 11,996) degrees of freedom is about 1.4 × 10⁻¹⁷⁴.

Move the sliders below to see where F comes from. More separation between the means grows the between bar; more spread inside the groups grows the within bar; more customers per group grows the between bar for the same gaps, because every customer counts in it.

How much does region matter?

A p-value of 1.4 × 10⁻¹⁷⁴ says the regions differ; it does not say by how much. The effect size for ANOVA is eta-squared, η², the share of the total variation that lies between the groups:

Region explains 6.5 % of the variance in days to payment: a medium effect by convention, and operationally significant for a collections function. The other 93.5 % is the variation between customers inside the same region.

What ANOVA assumes

• Independent observations: one row per customer, each in one region. Nothing in the data can check this; the design must.
• Roughly normal residuals: the customers' distances from their region's mean. Days to payment are right-skewed, but with 2,365 to 3,600 customers a region the means are close to normal, which is what the test needs.
• Similar variances across groups: Levene's test gives W = 0.75 and p = 0.52, so the four spreads are comparable and the classic ANOVA is valid.

If the variances clearly differ, especially with unequal group sizes, use Welch's ANOVA instead: it drops the equal-variance assumption, as Welch's t test does for two groups. The lab does not compute it; the Python is in the code below.

Which regions differ? Tukey's HSD

A significant F tells you "at least one group differs" and nothing about which. You then need a post-hoc test that compares all pairs while holding the family-wise error rate at 5 %. Tukey's HSD (honestly significant difference) is the standard choice. It is often said to need equal group sizes; its Tukey–Kramer form, the one scipy and statsmodels use, does not, which is why it is right for Helion's regions of 2,365 to 3,600 customers.

Each pair gets its own standard error from the within mean square, and its difference is compared with the studentized range, the distribution of the largest gap among k means that do not differ:

Worked for North against West: the means differ by 0.20 days, the standard error is \(\sqrt{11.34/2 \times (1/3600 + 1/2365)} = 0.063\) days, so q = 0.20 ÷ 0.063 = 3.15. The critical value for 4 groups at 5 % is 3.63, so the pair is not significant: p = 0.116. Every other pair clears it by far.

A significant F does not mean every pair differs

Note the asymmetry that catches people out: a significant omnibus F does not guarantee that any individual Tukey comparison will be significant, and it certainly does not mean every pair differs. Look at the North–West row. The F test was significant beyond any doubt, yet this pair cannot be told apart.

Reading "significant F" as "every region is different" leads straight to regional action plans aimed at differences that do not exist. The actionable finding is narrower and more useful: East lags the other three regions by 1.5 to 2.3 days, depending on the region, and warrants its own diagnosis.

Why it matters

A collections team has a budget for one regional programme. Six t tests would hand it a 26 % chance of chasing a phantom; the F test asks once, and Tukey's intervals say where to look: East, where customers pay up to 2.3 days later than the others. Without the post-hoc step, the team might split the budget four ways and waste most of it.

When this goes wrong

ANOVA's equal-variance assumption matters most when the groups also differ in size: if the smallest group is the most variable, the classic F raises too many false alarms, and Welch's ANOVA is the fix. Skew and outliers in small groups make the means unreliable; the rank-based Kruskal–Wallis test of chapter 14 is the fallback. And with 12,000 customers almost any difference is significant, so read η² and Tukey's intervals, not the p-value: a significant F with η² = 0.01 is a real but small effect.

In code

f_oneway, tukey_hsd and levene are in scipy.stats; statsmodels' pairwise_tukeyhsd gives the same Tukey intervals as a table. Excel's Data Analysis ToolPak runs "Anova: Single Factor"; it has no Tukey test, so the Excel line computes q for one pair, to compare with the critical value. If the variances differ, Welch's ANOVA is alexandergovern(north, south, east, west) in scipy.