Field note · May 27, 2025
Our most popular dataset is the broken one
Fourteen datasets, and the one people actually use is deliberately full of errors.
dirty-customers is a thousand rows of customer records broken in every way real data is broken: five date formats, mixed currency units, whitespace, inconsistent casing, sentinel nulls in five flavours, and 43 near-duplicates that differ only cosmetically.
It is by a distance the most downloaded thing on this site.
I did not expect that. I built it as a supporting asset for one lesson. It turns out that what people want is not to be told how to clean data — it is something broken to practise on, with a known answer.
That has changed how we build datasets. Every one now has a specific trap documented on its page:
- flight-delays: cancellations carry null delays, not zeros.
- saas-subscriptions: churn is right-censored; blank is not "did not churn".
- data-job-postings: salary missingness is informative.
- world-indicators: a genuine Simpson's paradox.
- movie-ratings: ratings are missing-not-at-random.
The dataset page names the trap and tells you what to try. That felt like giving away the answer at first. It is not — knowing that a paradox exists and finding it are different exercises, and the second one is where the learning is.
The near-duplicates in dirty-customers are the part people report enjoying most. Exact matching finds none of them. You have to normalise whitespace and casing first, which is a two-line fix that nobody thinks of until they have counted their customers twice.