Field note · December 17, 2023

Reading queue before trusting it in support-tickets

4,000 rows, 5 distinct values, and the one fact about queue that changes how you query it.

1 min read ·Data quality ·quality practice

Profiling queue on support-tickets before anyone builds anything on top of it.

4,000 rows, no nulls, 5 distinct values. Routing queue.

5 values, and they are not evenly spread: technical 32.1%, billing 24.0%, account 16.2%, onboarding 14.6%. The largest takes 32.1% on its own.

Read the distinct list rather than the distinct count. Casing differences and trailing whitespace produce values that look identical in a report and group separately in SQL, and the count will not show you that — select distinct queue order by 1 will, in about a second.

sql
select
  count(*)                as rows,
  count(queue)            as present,
  count(*) - count(queue) as nulls,
  count(distinct queue)   as distinct_values
from support_tickets;

Low cardinality, stable values — this is a column you can group by without thinking about it, and a reasonable candidate for a chart facet. Check the distinct list, not just the count, because a stray casing variant hides in the count and shows up in the group-by.

Where this bites: a group-by on this column produces 5 rows today. If it is a column an upstream system can add values to, it produces an unknown number tomorrow, and any dashboard laid out for 5 categories reflows without warning. Values arriving is a schema change that no schema check catches.

The grain is one row per ticket, which is the context every one of those numbers depends on. None of them survive a change of grain, which is why "profile the column" and "profile the table" are the same job. Full schema, and the CSV, on the dataset page.