Hypotheses and tails
What exactly am I testing, and in which direction? The null and the alternative, one tail or two, chosen before the data.
Hₐ: μ ≠ 500: reject H₀ at α = 0.05 if the average of 25 bottles is below 496.08 ml or above 503.92 ml.
The curve shows where the average of 25 bottles lands if the machine really fills 500 ml, with a standard error of 2 ml. Choose the question in words: different from 500, more than 500, or less than 500. The rejection region at α = 0.05 is shaded where that question points: below 496.08 and above 503.92 ml for "different from", above 503.29 ml for "more than", below 496.71 ml for "less than".
What you just saw
The curve is the null world for the bottles: where the average of 25 bottles lands if the machine really fills 500 ml on average. (With σ = 10 ml, that average varies with a standard error of 2 ml; chapter 3 explains why.) The shaded region is where an average would have to land for us to stop believing the machine is on target.
Which region depends on the question. "Is the fill different from 500?" shades both ends; "is it more than 500?" shades only the right; "is it less?" only the left. Everything in this chapter is about writing that question down precisely, and doing it before the data arrive.
Four words: population, sample, parameter, statistic
On the left, a cloud of dots stands for the population: every bottle the line fills, whose mean μ and standard deviation σ nobody knows. Some of its dots are picked out and drawn again on the right: the sample of 25 bottles that were weighed, whose mean x̄ = 504 ml and standard deviation s = 10 ml are known.
Four words are used precisely throughout this lab. Mixing them up is the commonest source of early confusion.
| Term | Meaning | Notation | Known? |
|---|---|---|---|
| Population | Every unit you want to draw a conclusion about: every bottle the line fills | – | No |
| Sample | The part you actually measured: the 25 bottles | – | Yes |
| Parameter | A number describing the population | μ, σ, p | No: it is what you are trying to learn |
| Statistic | The same number worked out from the sample | x̄, s, p̂ | Yes: you calculated it |
The two hypotheses
A test always sets up two competing statements about a parameter. They cannot both be true, and between them they cover every possibility, so ruling one out leaves only the other.
The null hypothesis, H₀, is the position of no effect, no difference or the claimed value: the status quo, the dull explanation, the thing you assume until the evidence forces you off it. It always contains an equality: =, ≤ or ≥. That is not an arbitrary convention. The null must name a definite value so that you can work out probabilities under it; you cannot say how surprising your data would be under a hypothesis as vague as "the mean is somewhere above 500".
The alternative hypothesis, Hₐ (also written H₁), is what you are gathering evidence for: an effect exists, the groups differ, the claim is wrong. It contains a strict inequality: ≠, > or <.
The courtroom, and where it breaks
The usual picture is a criminal trial. The court starts by assuming the defendant is not guilty (H₀). The prosecution presents evidence. If the evidence passes the standard of proof, the assumption is dropped and the verdict is guilty (reject H₀). If not, the verdict is not guilty, which is not the same as proven innocent.
That last point matters most, and chapter 3 comes back to it. But see where the picture breaks. A court weighs evidence by judgement, while a test works out an exact probability. And a court has one defendant, whereas an analyst runs many tests. Suppose an analyst runs forty tests a quarter on changes that in truth do nothing. At a 5 % threshold, about two of them will come out "guilty" by chance every quarter: eight innocents convicted a year. Chapter 18 deals with that.
Setting them up: the bottles
A maker claims that its bottles contain 500 ml on average. You suspect the filling machine has drifted, but you have no reason to think it drifted in a particular direction. Then:
Look at what went where. The maker's claim, 500 ml, went into the null, because it is the definite value. Your suspicion, drift, went into the alternative, because it is what you need evidence to establish. Beginners often swap them round and then find the test does something they did not expect.
One tail or two
The form of the alternative decides where the evidence has to fall to count against H₀. The figure at the top of the chapter shows the three cases:
- Two-tailed, Hₐ: μ ≠ 500. You care about a departure in either direction: a mean of 480 and one of 520 are both interesting. The rejection region sits in both tails, with α split between them: 2.5 % in each when α = 0.05.
- Right-tailed, Hₐ: μ > 500. You only care about an increase: a process change that raises throughput, a drug that improves outcomes. All 5 % of the region sits in the upper tail.
- Left-tailed, Hₐ: μ < 500. You only care about a decrease: under-filling, a defect rate that has fallen, a load time that has improved.
For the bottles, with a standard error of 2 ml, the average must land below 496.08 or above 503.92 ml to reject two-tailed at 5 %. One-tailed, it need only clear 503.29 ml (or fall below 496.71 ml). A one-tailed test is more powerful in the direction it looks, because it puts the whole of α in one tail.
Choose the tail before you see the data
Each dot is one experiment in a world where the machine is on target, placed at its z statistic. The honest analyst chose a two-tailed test in advance and rejects beyond ±1.96. The peeker looks at the data first and tests one tail in whichever direction they lean, which rejects beyond ±1.645. Two counters show the false alarms: the honest test settles near 5 %, the peeker's near 10 %.
The animation shows why the choice has to come first. It ran 1,000 experiments in a world where the machine is on target, so every rejection is a false alarm. The honest analyst chose a two-tailed test in advance: 49 false alarms, close to the 5 % promised. The peeker looked at which way the data leaned and then tested that tail alone: 98 false alarms, close to 10 %. Picking the tail after seeing the data quietly doubles the false-alarm rate.
Why this is true: Why peeking doubles the false alarms
The peeker tests the upper tail when z comes out positive and the lower tail when it comes out negative, each at the one-tailed cut-off. So the peeker rejects whenever |z| > 1.645.
If H₀ is true, z follows the standard normal curve, and the chance that |z| > 1.645 is 0.10: 5 % in each tail. The honest two-tailed test rejects only when |z| > 1.96, which happens with probability 0.05.
Why it matters
Writing H₀ and Hₐ down, with the tail, before the data exist is what makes the answer trustworthy. The bottling plant's quality manager acts on drift either way: under-filling short-changes customers, over-filling gives product away. So the test is two-tailed. A marketing team testing a new checkout may want to "prove it is better", but a worse checkout is exactly what they must catch: two-tailed again.
When this goes wrong
A one-tailed test is legitimate only when the opposite direction truly does not matter and that was decided in advance, in writing. A drug trial that can only license improvements can argue it; almost no business experiment can. And no choice of hypotheses rescues a vague question: "is the new page better?" must become a parameter (the conversion rate), a value (the old rate) and a direction before any test can answer it.
In code
| What it computes | Python (scipy, statsmodels) | Excel |
|---|---|---|
| Two-tailed t test of μ = 500 from the data | ttest_1samp(x, 500, alternative='two-sided') |
=T.DIST.2T(ABS(B1), 24) |
| Right-tailed, Hₐ: μ > 500 | ttest_1samp(x, 500, alternative='greater') |
=T.DIST.RT(B1, 24) |
| Left-tailed, Hₐ: μ < 500 | ttest_1samp(x, 500, alternative='less') |
=T.DIST(B1, 24, TRUE) |
| The z cut-off, two-tailed at α = 0.05 | norm.ppf(0.975) |
=NORM.S.INV(0.975) |
| The z cut-off, one-tailed at α = 0.05 | norm.ppf(0.95) |
=NORM.S.INV(0.95) |
scipy's alternative is chosen before you run the test, like the hypotheses. In Excel, B1 holds the t statistic from the data (chapter 6 computes it for the bottles): Excel has no one-sample t test function, so you compute t by formula and take its tail.
Quiz
Three questions on the chapter. Your best score is kept in this browser.