Correlation tests: Pearson and Spearman
H₀: ρ = 0 · H₁: ρ ≠ 0
r = 0.8474 · df = 78 · p = 3.80E-23
p = 3.80E-23 ≤ α = 0.05: reject H₀. The two variables move together more than chance alone would easily produce (r = 0.8474); correlation is not causation.
How to read the picture
The curve is the null world of t: how the correlation's t would vary if the true correlation were 0.
The vertical line is your t = r √(n − 2) / √(1 − r²).
The shaded tail is the p-value; the lighter region is where t must land to reject at α.
Below, r with Fisher's interval: an interval that excludes 0 goes with a rejection.
When both variables are numbers measured on the same units, the question is whether they move together. Pearson's r measures a straight-line relationship; Spearman's ρ is Pearson's r on the ranks, so it catches any one-directional relationship and shrugs off outliers. Each becomes a t statistic on n − 2 degrees of freedom.
The estimate and what the data show
The worked steps
Sums of squares and products
n = 80 · Sxx = 7058.4839 · Syy = 8322.7769 · Sxy = 6494.9494The correlation
r = Sxy / √(Sxx Syy) = 0.8474The t statistic
t = r √(n − 2) / √(1 − r²) = 0.8474 × √78 / √(1 − 0.8474²) = 14.0951Degrees of freedom
df = n − 2 = 78The p-value
p = 2 × P(T ≥ |14.0951|) = 3.80E-23Critical value at α, as a correlation
α = 0.05: r* = ±0.2199Share of variation explained
r² = 0.7181Fisher's interval for the correlation
95 %: [0.7713, 0.8996]
The same in code
- scipy
from scipy import stats result = stats.pearsonr(x, y, alternative='two-sided') result.statistic, result.pvalue, result.confidence_interval(0.95)- Excel
B1: =CORREL(A2:A81, B2:B81) B2: =B1*SQRT(78)/SQRT(1-B1^2) B3: =T.DIST.2T(ABS(B2), 78)
Formulas
S_xy adds the products of each pair's distances from the two means; t turns r into a test on n − 2 degrees of freedom, and Fisher's z gives the interval.
Try this
Outliers and ranks
Open the ten points with one outlier and switch between Pearson and Spearman.
Look for: One point far from the rest makes Pearson's r large and significant; on ranks it is only the largest value, and ρ stays modest. Plot the data before trusting r.
Small samples, wide intervals
Type five pairs that rise together, for example x = 1, 2, 3, 4, 5 and y = 2, 1, 4, 3, 5.
Look for: r = 0.8 looks strong, but with five pairs Fisher's interval runs from well below 0 to nearly 1 and the test does not reject.
Correlation is not pairing's whole story
Open Helion's agents with Pearson and read r, then open the two-means explorer's paired preset for the same agents.
Look for: The strong correlation between before and after is what makes the paired test so much sharper than treating the agents as two groups: the busy stay busy, and pairing takes that out.