Field note · February 4, 2024
The top of gdp_per_capita_usd is 1.4× the 99th percentile
A max 1.4× the 99th percentile, and the three different things that could be.
gdp_per_capita_usd on world-indicators: p99 is 98,996 and the max is 140,012.
The max is 1.4× the 99th percentile and 4.6 standard deviations above the mean. The median is 20,761, so the top of this column lives in a different regime from the middle of it.
select *
from world_indicators
order by gdp_per_capita_usd desc
limit 20;Run that before deciding anything. There are three things an extreme value can be, and they need opposite treatment: a data error, a real but rare event, or a different population that got mixed into the same table. Only the first is safe to drop, and you cannot tell them apart from the summary statistics — you have to look at the rows.
A ratio this modest is consistent with an ordinary heavy tail — nothing anomalous, just a distribution that is not symmetric.
Clipping at the 99th percentile is a decision to discard your most interesting rows. Sometimes right. Never automatic, and never silent — if a number in a report was computed on clipped data, the report has to say so.