Field note · February 27, 2025

Reading sessions before trusting it in ab-test-checkout

6,000 rows, 20 distinct values, and the one fact about sessions that changes how you query it.

1 min read ·Data quality ·quality practice

Profiling sessions on ab-test-checkout before anyone builds anything on top of it.

6,000 rows, no nulls, 20 distinct values. Sessions during the experiment window.

The five-number version: min 1, p10 1, median 2, p90 5, max 22. The mean is 3.

The p10 to p90 band — 1 to 5 — is where ordinary rows live, and it is the pair worth quoting when somebody asks what to expect. Min and max describe the two strangest rows in the table and nothing else; they are useful for spotting impossible values and misleading for everything else.

sql
select
  count(*)                   as rows,
  count(sessions)            as present,
  count(*) - count(sessions) as nulls,
  count(distinct sessions)   as distinct_values
from ab_test_checkout;

The mean sits 27% above the median, which is the whole story: anything that reports the average of sessions is reporting a number most rows are below. Put the median next to it or drop the mean.

Where this bites: a filter like sessions > 3 reads as "above average" and selects a minority of rows — a smaller minority than the phrase suggests to whoever asked for it. If the request was "the typical ones", the threshold they meant was 2.

The grain is one row per exposed user, which is the context every one of those numbers depends on. None of them survive a change of grain, which is why "profile the column" and "profile the table" are the same job. Full schema, and the CSV, on the dataset page.