Field note · June 24, 2024
r = 0.77 between gdp_per_capita_usd and life_expectancy
A measured correlation of 0.77 between two columns, what its square says, and the diagnostic that costs one line.
Measured on world-indicators: the Pearson correlation between gdp_per_capita_usd and life_expectancy over 720 rows is 0.77.
That is strong and positive. Squared, it says the linear relationship accounts for 58.8% of the variance in either column. That is the number worth quoting rather than r itself: 0.77 sounds like a much stronger claim than 58.8% does.
df[["gdp_per_capita_usd", "life_expectancy"]].corr(method="pearson")
# also worth running:
df[["gdp_per_capita_usd", "life_expectancy"]].corr(method="spearman")Run Spearman next to Pearson every time. They agree when the relationship is roughly linear and diverge when it is monotone but curved — and the gap between them is a free diagnostic that costs one line.
A correlation this size will get read as a mechanism. It is not one. Both columns here are generated from a shared underlying process, which is exactly the situation a confounder creates in real data: two things that move together because a third thing moves both.
Look at the scatter before quoting r. Anscombe's quartet is four datasets with identical correlation coefficients and nothing else in common, and the correlation explorer will draw this pair so you can see which case you are in.
Longer treatment in Correlation, confounding, and Simpson's paradox.