Chi-square: goodness of fit and independence

What you just saw

Helion's capacity plan assumed a contact mix of 46 % phone, 24 % chat, 18 % email and 12 % self-serve, and the call centre was staffed on it. During the experiment 1,600 customers got in touch. Does the mix match the plan?

The animation first drew the counts each channel actually received, then the counts the plan predicts for 1,600 contacts: 736 phone, 384 chat, 288 email and 192 self-serve. Then, one channel at a time, it measured the miss and added it to a running total. Email missed the most, 271 against 288, and even its contribution was only 1.003. The total, χ² = 1.35, landed far to the left of the critical value on the χ² curve: p = 0.72.

Here, unusually, a large p-value is the good news: the observed mix is consistent with the plan the staffing model was built on, so no re-plan is required.

Observed against expected

Chi-square tests work on counts of categories: never on means, percentages or measurements. If your table holds anything other than whole-number frequencies, you have the wrong test. Every chi-square test compares the observed count in each cell, O, with the expected count, E, that H₀ predicts:

In words: for each cell, square the miss (so misses in both directions count) and divide by the expected count (so each miss is judged against how many were expected there); then add the cells. Dividing by E is what makes a 10-count miss matter far more in a cell expecting 20 than in one expecting 2,000. For the phone channel:

If H₀ is true, χ² follows the χ² curve with the right degrees of freedom, and large values are the surprising ones: the p-value is the area to the right.

Goodness of fit: do the counts match a plan?

A goodness-of-fit test asks whether observed counts match a distribution fixed in advance. H₀ is the plan. The expected counts come from multiplying each hypothesised share by the total observed count, and the degrees of freedom are the number of categories less one (once three counts are known, the fourth is fixed by the total):

χ² = 1.35 on 3 degrees of freedom, against a critical value of 7.815 at α = 0.05: p = 0.72, we fail to reject H₀. Note the wording. This does not prove the forecast is correct, only that the data give no reason to abandon it. The distinction from chapter 3 matters as much in a capacity meeting as in a statistics exam.

Independence: does one category depend on another?

Helion's second counting question: is a customer's segment associated with how they pay? H₀ says the two are independent: the split of payment methods looks the same inside every segment. The expected counts then come from the table's margins alone, and the animation below fills them in cell by cell:

In words: if segment and payment method were unrelated, a cell's share of its row would be the column's share of the whole table. For Corporate customers paying by bank transfer: 975 Corporate customers, 2,624 bank-transfer payers, 12,000 in all, so 975 × 2,624 / 12,000 = 213.2 expected, against 570 observed. With 3 segments and 4 methods, df = 2 × 3 = 6.

The test of independence assumes independent observations, one row per customer (never test a table where the same person is counted twice), and expected counts of at least 5 in essentially every cell. Here the smallest expected count is 46.5, comfortably above 5, so the χ² approximation is safe. If cells are sparse, collapse categories or use Fisher's exact test (chapter 10).

The result: χ² = 1,298.3 on 6 degrees of freedom, p = 2.5 × 10⁻²⁷⁷. The association is real.

Where the association lives: the residuals

The standardised residuals show what the association is: Corporate customers concentrate heavily in bank transfer (+24.4; 58.5 % of them pay that way, against 14.1 % of Residential customers), while Residential customers concentrate in direct debit (+4.3) and card (+6.6), and avoid bank transfer (−15.2). That is the finding a collections strategy can be built on. A single organisation-wide "move everyone to direct debit" campaign would aim at the wrong behaviour for an entire segment. The p-value could never have said that; the residuals do.

How strong? Cramér's V

With 12,000 customers, even a weak association gives a huge χ². Cramér's V rescales χ² to run from 0 (no association) to 1 (each segment pays one way only), whatever the sample size:

V = 0.23: a real association of small-to-medium strength. Segment shapes how customers pay, but it is far from the whole story.

Why it matters

Counts of categories are everywhere in a business: contacts by channel, customers by segment, claims by region. The goodness-of-fit test tells a capacity planner whether the plan still holds; the test of independence, read through its residuals, tells a collections team which segment to target with which payment method. In both, the test is the start of the reading, not the end.

When this goes wrong

Chi-square tests need counts, one per independent unit: running one on percentages throws away the sample size and gives nonsense, and counting the same customer in two cells breaks independence. Small expected counts (below 5) make the χ² curve a poor approximation: collapse categories or use an exact test. And with large samples almost any association is significant, so read V and the residuals before anyone acts on a p-value like 2.5 × 10⁻²⁷⁷.

In code

chisquare and chi2_contingency are in scipy.stats. Excel's CHISQ.TEST needs the expected table built by formula, returns only the p-value, and picks its own degrees of freedom: k − 1 for one row, which is wrong when a parameter was estimated from the data. G1 holds χ².