Explore

Equivalence testing (TOST)

H₀: GMR ≤ 0.818731 or GMR ≥ 1.221403 · H₁: 0.818731 < GMR < 1.221403

t = 3.1943 · df = 4554.5385 · p = 0.0007

p = 0.0007 ≤ α = 0.05: equivalence is shown. Equivalence is shown: the interval [0.8709, 0.9931] lies inside the margin [0.818731, 1.221403], so the difference (0.93) is smaller than the one that matters.

How to read the picture

The curve is the null world of the one-sided test that binds, the one with the larger p-value; the TOST p-value is that p.
Below, the 1 − 2α interval against the equivalence margin: inside the band, equivalence is shown.
On the log scale the difference of log means is a ratio of geometric means, not of arithmetic means.

Your data

n₁ = 2062

n₁ (size of group 1)

x̄₁ = 7.634082

x̄₁ (mean of group 1; of the logs on the log scale)

s₁ (standard deviation of group 1)

n₂ = 2601

n₂ (size of group 2)

x̄₂ = 7.706665

x̄₂ (mean of group 2; of the logs on the log scale)

s₂ (standard deviation of group 2)

Scale

The raw values: a difference of means

Logged values: a ratio of geometric means

Lower margin (a difference, or a ratio on the log scale)

Upper margin (a difference, or a ratio on the log scale)

Variances

Welch: unequal variances (the default)

Pooled: equal variances assumed

Significance level α

α = 0.1

α = 0.05

α = 0.01

α = 0.001

To show that two means are close enough to treat as the same, turn the question round: H₀ says the difference is at least as large as a margin that matters, and two one-sided t tests try to reject both ends. Equivalence is shown when both reject, which is when the 90 % interval (for α = 0.05) lies inside the margin. The margin comes from the decision, never from the data.

Not sure this is the right test? Ask the test chooser

The estimate and what the data show

On the log scale this is equivalence of the geometric means. If the decision uses the arithmetic mean, as a tariff does, test that mean instead: the two can disagree (correction C29).

The worked steps

The difference of the log means
x̄₁ − x̄₂ = 7.6341 − 7.7067 = -0.07258
Standard error (Welch)
SE = 0.03989 · df = 4554.5385
The margins in logs
Δ_L = ln 0.818731 = -0.2 · Δ_U = ln 1.221403 = 0.2
Test 1: is the difference above the lower margin?
t_L = (-0.07258 − (-0.2)) / 0.03989 = 3.1943 · p_L = P(T ≥ 3.1943) = 0.0007
Test 2: is the difference below the upper margin?
t_U = (-0.07258 − 0.2) / 0.03989 = -6.8337 · p_U = P(T ≤ -6.8337) = 4.68E-12
The TOST p-value
p = max(0.0007, 4.68E-12) = 0.0007
The matching interval
90 %: -0.07258 ∓ 1.6452 × 0.03989 = [-0.1382, -0.006959]
As a ratio of geometric means, beside the margin
e^-0.07258 = 0.93 · [0.8709, 0.9931] · [0.818731, 1.221403]

The same in code

statsmodels, from the data

import numpy as np
from statsmodels.stats.weightstats import ttost_ind

ttost_ind(np.log(a), np.log(b), -0.2, 0.2, usevar='unequal')

scipy, from the summaries

from scipy import stats

t_low = (-0.07258 - (-0.2)) / 0.03989
t_upp = (-0.07258 - 0.2) / 0.03989
p = max(stats.t.sf(t_low, 4554.5385), stats.t.cdf(t_upp, 4554.5385))

Excel

B1: =T.DIST.RT(3.1943, 4554.5385)
B2: =T.DIST(-6.8337, 4554.5385, TRUE)
B3: =MAX(B1, B2)

Excel has no equivalence test: the two one-sided p-values come from T.DIST, and the larger is the TOST p-value. T.DIST functions cut a fractional df down to a whole number.

Formulas

The first test asks whether the difference is above the lower margin, the second whether it is below the upper one; equivalence needs both, so the larger p-value decides.

Try this

Not significant is not equivalent

Open the small study: the two means differ by 1.3 ml, not significantly, yet equivalence within ±3 ml is not shown either.

Look for: The interval runs well past both margins. A test that fails to find a difference has not shown that there is none; only an interval inside the margin does.
The margin decides

Open Meridian H7 and narrow the margin to 0.9 and 1.11 (about ±0.10 in logs).

Look for: The interval does not move, but it no longer fits inside the band. The margin is a business decision, fixed before the data are seen.
More data, a narrower interval

Open the two filling lines and raise both sample sizes from 40 to 400.

Look for: The means and spreads stay, the interval narrows around −1.2 ml, and it fits inside ±3 ml: equivalence needs enough data to pin the difference down.