Proportions: one and two, z and exact
H₀: p = 0.95 · H₁: p ≠ 0.95
z = -1.026 · p = 0.3049
p = 0.3049 > α = 0.05: H₀ is not rejected. The data do not rule out a true proportion of 0.95 (the sample gave 0.94), but that is not proof that it is 0.95.
How to read the picture
For the z tests, the curve is the null world of z; for the exact tests, the bars are the probability of every possible count if H₀ were true.
The vertical line is your result: z, or the observed count.
The shaded area or the highlighted bars are the p-value: the results at least as extreme as yours.
Below, the proportion or the gap with its interval.
Your data
Test
Alternative hypothesis
Significance level α
A proportion is a count of yeses out of n. Test one against a target (a courier's promise, a tariff's large-loss rate) or compare two groups (the old portal and Portal v2). The z test uses the normal curve; the exact tests count the outcomes themselves, which is what small counts need.
The estimate and what the data show
The worked steps
The sample proportion
p̂ = k / n = 470 / 500 = 0.94Is the normal curve safe?
n p₀ = 475 · n (1 − p₀) = 25Standard error under H₀
SE = √(p₀ (1 − p₀) / n) = √(0.95 × 0.05 / 500) = 0.009747The z statistic
z = (p̂ − p₀) / SE = (0.94 − 0.95) / 0.009747 = -1.026The p-value
p = 2 × P(Z ≥ |-1.026|) = 0.3049Critical value at α
α = 0.05: z* = ±1.96Wilson interval for p
95 %: [0.9156, 0.9577]Effect size (Cohen's h)
h = 2 asin √p̂ − 2 asin √p₀ = -0.04391
The same in code
- scipy, from the counts
from scipy import stats z = (470 / 500 - 0.95) / (0.95 * (1 - 0.95) / 500) ** 0.5 p = 2 * stats.norm.sf(abs(z))- statsmodels (prop_var=p₀ puts p₀ in the standard error)
from statsmodels.stats.proportion import proportions_ztest proportions_ztest(470, 500, value=0.95, alternative='two-sided', prop_var=0.95)- Excel
B1: =(470/500-0.95)/SQRT(0.95*(1-0.95)/500) B2: =2*(1-NORM.S.DIST(ABS(B1), TRUE))
Formulas
The standard error uses p₀, the proportion H₀ claims; the normal curve is a good fit when n p₀ and n (1 − p₀) are both at least 10.
Try this
z against exact
Open the tiny trial with Fisher's test, then switch to the two-proportion z test.
Look for: With 7 successes in all, the z test's normal curve is a poor fit and gives a smaller p-value than the exact count. Use an exact test when the expected counts are small.
Same gap, more customers
Open the small pilot and raise both n from 1,000 to 4,000, keeping the rates (multiply both counts by 4).
Look for: The gap of 2.8 points stays, the standard error halves, and the pilot's 'not significant' becomes significant: the pilot was too small, not the effect.
A one-sided question
Open the courier's 470 of 500 and switch the alternative to 'less': is the rate below the promise?
Look for: The one-sided p-value is half the two-sided one here, and still above 0.05: 94 % is not shown to be below 95 % by 500 parcels.