Glossary

The terms the chapters use, each with a short definition and a link to the section that explains it.

Showing 80 of 80 terms.

Alternative hypothesis (Hₐ)

The statement the analyst seeks evidence for. It contains a strict inequality: ≠, > or <. Taught in chapter 2: Hypotheses and tails

ANOVA

Analysis of variance. Compares three or more means using the ratio of the between-group to the within-group variance, F. Taught in chapter 13: One-way ANOVA and after

Benjamini–Hochberg procedure

Compares the i-th smallest of m p-values with i·α/m and rejects all up to the largest that passes. Controls the false discovery rate, not the family-wise error rate. Taught in chapter 18: Multiple testing, pre-registration and p-hacking

Bonferroni correction

Tests each of m hypotheses at α/m, holding the family-wise error rate at α. Simple and conservative. Taught in chapter 18: Multiple testing, pre-registration and p-hacking

Claim frequency

Claims per unit of exposure, such as claims per vehicle-year: how often policies claim. Taught in chapter 20: A risk review end to end: the Meridian finale

Claim severity

The cost of a claim once it happens, averaged over claims, not over policies. Taught in chapter 20: A risk review end to end: the Meridian finale

Clopper–Pearson interval

The exact confidence interval for a proportion: the proportions the exact binomial test would not reject. It covers the true value at least as often as promised. Taught in chapter 9: One proportion

Cohen's d

A standardised effect size for means: the difference divided by a standard deviation, so it does not depend on the sample size. By convention 0.2 is small, 0.5 medium and 0.8 large. Taught in chapter 6: One mean: z and t, one complete test

Cohen's h

A standardised effect size for two proportions: 2 arcsin √p₁ − 2 arcsin √p₂. Taught in chapter 10: Two proportions

Confidence interval

A range of parameter values consistent with the data at a stated confidence level. A two-sided test rejects exactly the values outside it (for a mean's z and t). Taught in chapter 5: Tests and confidence intervals

Correlation coefficient (Pearson's r)

A number from −1 to 1 measuring how closely two numerical variables follow a straight line. Its test asks whether the population correlation is 0. Taught in chapter 15: Which test? The map, the assumptions, and correlation

Cramér's V

The strength of the association in a table of counts, from 0 (none) to 1 (complete), whatever the sample size. Taught in chapter 11: Chi-square: goodness of fit and independence

Critical value

The boundary of the rejection region on the test statistic's scale, such as ±1.96 for a two-tailed z test at α = 0.05. Taught in chapter 3: The p-value

Degrees of freedom

The number of independent pieces of information available for estimating variability: n − 1 for a one-sample t. Taught in chapter 5: Tests and confidence intervals

Discordant pairs

In paired yes-or-no data, the cases where the two measurements disagree. McNemar's test uses only these. Taught in chapter 20: A risk review end to end: the Meridian finale

Effect size

A measure of the magnitude of a difference that does not depend on the sample size (Cohen's d, Cohen's h, Cramér's V, η²), or the raw difference in real units. Taught in chapter 16: Effect size, intervals and "not significant"

Epsilon-squared (ε²)

The Kruskal–Wallis test's effect size: roughly the share of the variation in ranks that lies between the groups. Taught in chapter 14: When the assumptions fail: ranks and permutations

Equivalence margin

The largest difference a decision would treat as unimportant, fixed in advance; an equivalence test asks whether the true difference lies inside it. Taught in chapter 20: A risk review end to end: the Meridian finale

Equivalence test (TOST)

A test that can show two quantities differ by less than a margin fixed in advance, by two one-sided tests against the margin's two edges. Taught in chapter 16: Effect size, intervals and "not significant"

Eta-squared (η²)

ANOVA's effect size: the share of the total variation that lies between the groups, SS between ÷ SS total. Taught in chapter 13: One-way ANOVA and after

Exact binomial test

Tests one proportion by adding the exact binomial probabilities of every outcome at least as extreme as the one observed, with no normal approximation. Taught in chapter 9: One proportion

Expected count

The count a cell would hold if H₀ were true: a plan's share times the total, or row total × column total ÷ grand total. Taught in chapter 11: Chi-square: goodness of fit and independence

Exposure

How much risk a policy carried, in time: earned vehicle-years. A policy in force for four months has a third of a vehicle-year. Taught in chapter 20: A risk review end to end: the Meridian finale

F test

Compares two variances by their ratio, read on the F curve. Exact only for normal data, and very sensitive to anything else. Taught in chapter 12: Variances: χ², F and Levene

Fail to reject

What a test concludes when p ≥ α: the evidence was not strong enough to reject H₀. It never shows that H₀ is true. Taught in chapter 3: The p-value

False discovery rate

The expected share of the rejected hypotheses that are in fact true nulls: the false leads among the discoveries. Taught in chapter 18: Multiple testing, pre-registration and p-hacking

Family-wise error rate

The chance of at least one false positive across a whole family of tests. It grows with every test added unless a correction such as Holm's or Bonferroni's holds it at α. Taught in chapter 13: One-way ANOVA and after

Fisher's exact test

Tests a 2 × 2 table exactly from its counts, given its row and column totals: the choice when a cell holds few successes or failures. Taught in chapter 10: Two proportions

Fligner–Killeen test

A rank-based test of equal spreads: it ranks each value's distance from its group's median and compares the groups' average scores. A second opinion to Levene's test when the data are far from normal. Taught in chapter 12: Variances: χ², F and Levene

Garden of forking paths

The many analysis choices a data set allows; walking them after the data turns a 5 % error rate into near-certainty of a false win. Taught in chapter 18: Multiple testing, pre-registration and p-hacking

Geometric mean ratio

The ratio of two geometric means, found by exponentiating the difference of mean logs: how much the typical value changed, as a factor. Taught in chapter 20: A risk review end to end: the Meridian finale

Goodness-of-fit test

A chi-square test of whether observed counts match a distribution fixed in advance, on the number of categories less one degrees of freedom. Taught in chapter 11: Chi-square: goodness of fit and independence

Holm's method

A step-down version of Bonferroni: the smallest p-value is compared with α/m, the next with α/(m − 1), and so on until one fails. Controls the family-wise error rate and never rejects fewer than Bonferroni. Taught in chapter 18: Multiple testing, pre-registration and p-hacking

Independence

The assumption that one observation tells you nothing about another: no unit counted twice, no clusters. Established by the design and the data grain, never by a test. Taught in chapter 15: Which test? The map, the assumptions, and correlation

Kruskal–Wallis test

The rank-based alternative to one-way ANOVA: asks whether at least one of several groups tends to give larger values. Taught in chapter 14: When the assumptions fail: ranks and permutations

Large loss

A claim above a set threshold (here £25,000), priced in a tariff through its own loading. Taught in chapter 20: A risk review end to end: the Meridian finale

Levene's test

Compares spreads through an ANOVA on each value's distance from its group's centre (the median by default): robust to data that are not normal. Taught in chapter 12: Variances: χ², F and Levene

Loss cost

The cost of claims per unit of exposure, such as pounds per vehicle-year: claim frequency times claim severity. Taught in chapter 20: A risk review end to end: the Meridian finale

Mann–Whitney U test

The rank-based alternative to the two-sample t test: asks whether values from one group tend to be larger than values from the other. Taught in chapter 14: When the assumptions fail: ranks and permutations

McNemar's test

A test for two yes-or-no outcomes measured on the same cases: do the disagreements split evenly? A sign test on the discordant pairs. Taught in chapter 20: A risk review end to end: the Meridian finale

Minimum detectable effect (MDE)

The smallest effect a design could reliably detect at a stated power, usually 80 %. Taught in chapter 4: Errors and power

Null hypothesis (H₀)

The statement of no effect, no difference, or the claimed value. It contains an equality, so probabilities can be worked out under it. Taught in chapter 2: Hypotheses and tails

Null world

The world where nothing has changed and H₀ is true: what the data would look like from luck alone. A test asks how often it produces a result at least as extreme as yours. Taught in chapter 1: The null world

Observed power

Power computed after a test at the effect the sample happened to show. It restates the p-value and adds no evidence; plan power before the data instead. Taught in chapter 17: Power and sample size

One-sample t test

Compares one sample's mean with a hypothesised value when σ is unknown: t = (x̄ − μ₀) / (s/√n) on n − 1 degrees of freedom. Taught in chapter 6: One mean: z and t, one complete test

One-sample z test

Compares one sample's mean with a hypothesised value when σ is known from a long record: z = (x̄ − μ₀) / (σ/√n), read on the standard normal curve. Taught in chapter 3: The p-value

One-tailed test

A test whose alternative points one way (> or <): all of α sits in one tail. Legitimate only when the direction is chosen before the data. Taught in chapter 2: Hypotheses and tails

p-hacking

Choosing the analysis (metric, segment, tail, α, outliers, stopping point) after seeing the data until the result is significant, often in good faith. Taught in chapter 18: Multiple testing, pre-registration and p-hacking

p-value

The probability of a result at least as extreme as the one observed, assuming H₀ is true. Taught in chapter 3: The p-value

Paired t test

A one-sample t test on the differences within pairs (the same unit measured twice, or matched units); it removes the variation between units. Taught in chapter 8: Paired samples

Parameter

A number describing the population, such as its mean μ, its standard deviation σ or a proportion p. Hypotheses are always about parameters. Taught in chapter 2: Hypotheses and tails

Peeking

Checking a test repeatedly as data arrive and stopping at the first significant result. It inflates the false-positive rate far above α. Taught in chapter 18: Multiple testing, pre-registration and p-hacking

Permutation test

A test that builds the null distribution by shuffling the group labels many times and recomputing the statistic; it needs no assumption about the data's shape. Taught in chapter 14: When the assumptions fail: ranks and permutations

Population

Every unit you want to draw a conclusion about: all your customers, all the bottles a line fills. Its numbers (parameters) are usually unknown. Taught in chapter 2: Hypotheses and tails

Post-hoc test

A follow-up comparison after a significant omnibus test, adjusted for multiplicity, that says which groups differ (for example Tukey's HSD). Taught in chapter 13: One-way ANOVA and after

Power (1 − β)

The probability of detecting an effect that genuinely exists. It grows with the sample, the effect and α, and shrinks with noise. Convention targets 0.80. Taught in chapter 4: Errors and power

Practical significance

Whether an effect is large enough to matter for a decision, judged against the smallest effect worth acting on, not by the p-value. Taught in chapter 16: Effect size, intervals and "not significant"

Pre-registration

Committing the hypotheses, tails, α, primary metric, sample size and corrections to a timestamped record before the data are collected. Taught in chapter 18: Multiple testing, pre-registration and p-hacking

Quality gate

A check on the data (one row per unit, no gaps, plausible values) that stops the analysis when it fails, before any hypothesis is tested. Taught in chapter 19: An A/B test end to end: the Helion readout

Rank-based test

A test that replaces each value by its rank, so it depends only on the order of the values: robust to skew and outliers (Mann–Whitney, Wilcoxon, Kruskal–Wallis). Taught in chapter 14: When the assumptions fail: ranks and permutations

Rate ratio

One group's event rate per unit of exposure divided by another's; 0.79 reads as a 21 % lower rate. Taught in chapter 20: A risk review end to end: the Meridian finale

Rejection region

The set of test-statistic values that lead to rejecting H₀: the most extreme α of the null distribution. Taught in chapter 3: The p-value

Sample

The part of the population you actually measured. Its numbers (statistics) are known, and they wobble from sample to sample. Taught in chapter 2: Hypotheses and tails

Sample ratio mismatch (SRM)

An observed split between the arms of an experiment that is unlikely under the planned one: a sign that the assignment, the logging or a filter is broken, checked with a χ² goodness-of-fit test. Taught in chapter 19: An A/B test end to end: the Helion readout

Sample size calculation

Working out, before a study, how many observations give the target power (usually 80 %) against the smallest effect worth acting on. Taught in chapter 17: Power and sample size

Significance level (α)

The accepted probability of a Type I error, chosen before collecting data. Reject H₀ when p < α. Taught in chapter 3: The p-value

Spearman's rank correlation

Pearson's r computed on ranks: measures whether two variables move in the same direction, whatever the shape of the relationship. Taught in chapter 15: Which test? The map, the assumptions, and correlation

Standard error

The standard deviation of a sample statistic across repeated samples, such as σ/√n for a mean. The denominator of every test statistic. Taught in chapter 3: The p-value

Standardised residual

(O − E) / √E for one cell of a chi-square table. Beyond about ±2 the cell contributes materially to the association. Taught in chapter 11: Chi-square: goodness of fit and independence

Statistic

The same kind of number worked out from the sample, such as x̄, s or p̂. You know it; a hypothesis about it is meaningless. Taught in chapter 2: Hypotheses and tails

t distribution

Student's t distribution: the null curve of a t statistic when σ is estimated by s. Flatter than the normal with heavier tails, it approaches the normal as the degrees of freedom grow. Taught in chapter 6: One mean: z and t, one complete test

Test of independence

A chi-square test of whether two categorical variables are associated; the expected counts come from the table's margins. Taught in chapter 11: Chi-square: goodness of fit and independence

Test statistic

Signal divided by noise: the observed departure from H₀, expressed in standard errors. Taught in chapter 3: The p-value

Tukey's HSD

Tukey's honestly significant difference: compares every pair of group means after ANOVA while holding the family-wise error rate at α. Its Tukey–Kramer form handles unequal group sizes. Taught in chapter 13: One-way ANOVA and after

Two-proportion z test

Compares two groups' rates, using the pooled rate for the standard error because H₀ says the rates are equal. The workhorse of A/B testing. Taught in chapter 10: Two proportions

Two-tailed test

A test whose alternative is "different from" (≠): evidence in either direction counts, and α is split between the two tails. Taught in chapter 2: Hypotheses and tails

Type I error

Rejecting a true H₀: a false positive, crying wolf. Its probability is α. Taught in chapter 4: Errors and power

Type II error

Failing to reject a false H₀: a false negative, missing the wolf. Its probability is β. Taught in chapter 4: Errors and power

Welch's t test

The two-sample t test that does not assume equal variances: each group keeps its own variance in the standard error. The sensible default for two independent means. Taught in chapter 7: Two independent means

Wilcoxon signed-rank test

The rank-based alternative to the one-sample and paired t tests: ranks the sizes of the differences and compares the positive with the negative ones. Taught in chapter 14: When the assumptions fail: ranks and permutations