Archive
2025
365 field notes published in 2025.
December 31
- data-job-postings.country: 7 values, 69.5% in the top three Visualization
- The average duration min is not the typical one Statistics
- Keeping one row per key in movie-ratings Analytics engineering
- Is account_id + plan the grain of saas-subscriptions? Analytics engineering
- Check the blanks before you drop them Statistics
- Five minutes with duration_min in ride-hail-trips Data quality
- Reading 3 columns instead of 7 Data platform
- Before you clip the outliers Statistics
- The overall average hides 4 different numbers Analytics practice
- A essentially none correlation, and what it is not Statistics
- Correction: Anti-join Analytics engineering
- Charting line_no by channel Visualization
- What SQL Playground will not do Visualization
- What is deliberately wrong with ab-test-checkout Data quality
- Course note — When not to build a model Machine learning
- Year three Analytics practice
- Median beside the mean, in short Statistics
- Is subject_id + sex the grain of clinical-trial? Analytics engineering
- ordered_at runs 2023-01-01 to 2024-12-30 Data engineering
- The category that eats the chart Visualization
- Finding the gaps in data-job-postings Analytics engineering
- 9.9% of week12_score is missing Statistics
- The average line no is not the typical one Statistics
- Five minutes with account_id in saas-subscriptions Data quality
- The arithmetic behind a cheap query Data platform
- Before you clip the outliers Statistics
- bmi by arm: a 0% spread Analytics practice
- A weak correlation, and what it is not Statistics
- Correction: Lookback window Data engineering
- Charting salary_min_usd by city Visualization
- Why Data Explorer works the way it does Visualization
November 30
- An exercise on sensor-telemetry Data quality
- Cutting a lesson down — Testing data like you test code Data quality
- The category that eats the chart Visualization
- Comparing a period to the one before it Analytics engineering
- Idempotent window replace, in short Data engineering
- Fifteen minutes a month on the warehouse bill Data platform
- The time coverage of flight-delays Data engineering
- Is subject_id + site the grain of clinical-trial? Analytics engineering
- The average first response min is not the typical one Statistics
- Reading reading_at before trusting it in sensor-telemetry Data quality
- Reading 3 columns instead of 10 Data platform
- The top of no2_ppb is 1.2× the 99th percentile Statistics
- salary_max_usd by city: a 427% spread Analytics practice
- A weak correlation, and what it is not Statistics
- Correction: Difference-in-differences Experimentation
- Charting release_year by genre Visualization
- Distribution Lab works offline Visualization
- An exercise on flight-delays Data quality
- Course note — Window functions, properly Analytics engineering
- Is order_id + line_no the grain of retail-orders? Analytics engineering
- Conditional pivot, in short Analytics engineering
- The time coverage of support-tickets Data engineering
- The category that eats the chart Visualization
- The average units is not the typical one Statistics
- Sessions out of raw events in ab-test-checkout Analytics engineering
- Five minutes with subject_id in clinical-trial Data quality
- Reading 3 columns instead of 10 Data platform
- The top of discount_pct is 1.1× the 99th percentile Statistics
- seats by plan: a 21,443% spread Analytics practice
- A essentially none correlation, and what it is not Statistics
October 31
- Correction: Conditional pivot Analytics engineering
- Charting first_response_min by queue Visualization
- The chart rules behind Correlation Explorer Visualization
- The question that kills the most projects Analytics practice
- The category that eats the chart Visualization
- An exercise on data-job-postings Data quality
- Cutting a lesson down — Idempotency, backfills, and yesterday arriving twice Data engineering
- As-of feature computation, in short Machine learning
- Comparing a period to the one before it Analytics engineering
- The time coverage of saas-subscriptions Data engineering
- Counting rows is not testing grain Analytics engineering
- Reading 3 columns instead of 8 Data platform
- The average fare usd is not the typical one Statistics
- Five minutes with city in city-air-quality Data quality
- The top of baseline_score is 1.1× the 99th percentile Statistics
- humidity_pct by status: a 4% spread Analytics practice
- r = 0.10 between sched_dep_hour and arr_delay_min Statistics
- Correction: Baseline before model Machine learning
- Charting age by arm Visualization
- Correlation Explorer works offline Visualization
- World development indicators, in one paragraph Data quality
- Reader question — Window functions, properly Analytics engineering
- Deduplicate by rank, in short Analytics engineering
- exposed_on runs 2024-03-04 to 2024-03-31 Data engineering
- clinical-trial.sex: 2 values, 100.0% in the top three Visualization
- A running total that does not lie about ties Analytics engineering
- Reading flight_no before trusting it in flight-delays Data quality
- Is reading_at + status the grain of sensor-telemetry? Analytics engineering
- The average gdp per capita usd is not the typical one Statistics
- Before you clip the outliers Statistics
- sched_dep_hour by carrier: a 3% spread Analytics practice
September 30
- Four charts I deleted from this site Visualization
- A essentially none correlation, and what it is not Statistics
- Correction: Median beside the mean Statistics
- Charting sessions by device Visualization
- What Regression Lab will not do Visualization
- Ride-hail trips, in one paragraph Data quality
- Course note — Designing an experiment that can succeed Experimentation
- Pattern: Median beside the mean Statistics
- The time coverage of movie-ratings Data engineering
- The category that eats the chart Visualization
- Keeping one row per key in ride-hail-trips Analytics engineering
- Five minutes with origin in flight-delays Data quality
- The average seats now is not the typical one Statistics
- Is device_id + reading_at the grain of sensor-telemetry? Analytics engineering
- Before you clip the outliers Statistics
- dep_delay_min by dest: a 38% spread Analytics practice
- Charting seats_now by plan Visualization
- A strong correlation, and what it is not Statistics
- Correction: Date spine Analytics engineering
- Experiment Calculator works offline Visualization
- churned_at runs 2023-03-17 to 2026-01-24 Data engineering
- What is deliberately wrong with ride-hail-trips Data quality
- Course note — Writing SQL people can read, and reading the plan Analytics engineering
- Gaps and islands, in short Analytics engineering
- saas-subscriptions.region: 4 values, 93.3% in the top three Visualization
- Comparing a period to the one before it Analytics engineering
- Is user_id + movie_id the grain of movie-ratings? Analytics engineering
- The average co2 tonnes per capita is not the typical one Statistics
- The top of o3_ppb is 1.2× the 99th percentile Statistics
- Five minutes with revenue_usd in retail-orders Data quality
August 31
- r = -0.00 between vibration_mm_s and rpm Statistics
- The overall average hides 3 different numbers Analytics practice
- Charting no2_ppb by city Visualization
- Why Distribution Lab works the way it does Visualization
- Factory sensor telemetry, in one paragraph Data quality
- No runbook, no page Data platform
- Cutting a lesson down — Rows, grain, and the shape of a dataset Analytics practice
- Counting your tests, in short Experimentation
- rated_at runs 2023-01-01 to 2025-06-19 Data engineering
- The category that eats the chart Visualization
- The average surge multiplier is not the typical one Statistics
- A running total that does not lie about ties Analytics engineering
- Counting rows is not testing grain Analytics engineering
- What arr_delay_min actually contains in flight-delays Data quality
- The top of duration_min is 1.3× the 99th percentile Statistics
- discount_pct by country: a 14% spread Analytics practice
- r = 0.75 between life_expectancy and co2_tonnes_per_capita Statistics
- Charting rating by genre Visualization
- What Experiment Calculator will not do Visualization
- The time coverage of data-job-postings Data engineering
- An exercise on dirty-customers Data quality
- Course note — Schema drift, and the contract that prevents it Data engineering
- The left join that became an inner join, in short Analytics engineering
- data-job-postings.city: 10 values, 41.8% in the top three Visualization
- passengers: mean 2, median 1 Statistics
- The top of unit_price_usd is 1.7× the 99th percentile Statistics
- A running total that does not lie about ties Analytics engineering
- Counting rows is not testing grain Analytics engineering
- Five minutes with passengers in ride-hail-trips Data quality
- discount_pct by category: a 16% spread Analytics practice
- r = 0.04 between temp_c and price_eur_mwh Statistics
July 31
- Charting pm25 by city Visualization
- Checkout A/B test, in one paragraph Data quality
- The CLT does not always save you Statistics
- Data Explorer works offline Visualization
- Cutting a lesson down — Joins that do not fan out Analytics engineering
- Cohort retention triangle, in short Analytics practice
- ab-test-checkout.device: 3 values, 100.0% in the top three Visualization
- Comparing a period to the one before it Analytics engineering
- Counting rows is not testing grain Analytics engineering
- The average unit price usd is not the typical one Statistics
- What site actually contains in sensor-telemetry Data quality
- The top of mrr_usd is 1.6× the 99th percentile Statistics
- distance_km by payment_type: a 3% spread Analytics practice
- r = 0.72 between load_mw and price_eur_mwh Statistics
- Charting year by region Visualization
- Why Experiment Calculator works the way it does Visualization
- City air quality, in one paragraph Data quality
- Comparing a period to the one before it Analytics engineering
- Course note — The anatomy of a silent failure Data quality
- Contract check at the boundary, in short Data quality
- city-air-quality.city: 4 values, 75.0% in the top three Visualization
- Counting rows is not testing grain Analytics engineering
- dep_delay_min: mean 10, median 5 Statistics
- What signed_up_on actually contains in saas-subscriptions Data quality
- Before you clip the outliers Statistics
- The overall average hides 4 different numbers Analytics practice
- r = -0.36 between pm10 and o3_ppb Statistics
- The category that eats the chart Visualization
- Charting year by country Visualization
- What Correlation Explorer will not do Visualization
- What is deliberately wrong with support-tickets Data quality
June 30
- Counting rows is not testing grain Analytics engineering
- Cutting a lesson down — Orchestration, and what "it runs every night" costs Data platform
- Lookback window, in short Data engineering
- A running total that does not lie about ties Analytics engineering
- The average seats is not the typical one Statistics
- What co2_tonnes_per_capita actually contains in world-indicators Data quality
- We turned off 80% of our alerts and caught more incidents Data platform
- Before you clip the outliers Statistics
- salary_max_usd by seniority: a 162% spread Analytics practice
- A essentially none correlation, and what it is not Statistics
- Charting revenue_usd by variant Visualization
- The chart rules behind Experiment Calculator Visualization
- Customer support tickets, in one paragraph Data quality
- Cutting a lesson down — Causal claims when you cannot randomise Experimentation
- Fan-out join, in short Analytics engineering
- The category that eats the chart Visualization
- Comparing a period to the one before it Analytics engineering
- distance_km: mean 4.83, median 3.20 Statistics
- What pm25 actually contains in city-air-quality Data quality
- Is resolved_at + priority the grain of support-tickets? Analytics engineering
- The top of humidity_pct is 1.1× the 99th percentile Statistics
- fare_usd by payment_type: a 4% spread Analytics practice
- A strong correlation, and what it is not Statistics
- Charting flight_no by carrier Visualization
- The chart rules behind SQL Playground Visualization
- An exercise on support-tickets Data quality
- Cutting a lesson down — Describing a column without lying Statistics
- Counting rows is not testing grain Analytics engineering
- Pattern: Date spine Analytics engineering
- world-indicators.region: 5 values, 70.0% in the top three Visualization
May 31
- Sessions out of raw events in sensor-telemetry Analytics engineering
- sched_dep_hour by origin: a 4% spread Analytics practice
- Five minutes with opened_at in support-tickets Data quality
- line_no: mean 2, median 1 Statistics
- Our most popular dataset is the broken one Data quality
- Before you clip the outliers Statistics
- A essentially none correlation, and what it is not Statistics
- Charting first_response_min by channel Visualization
- What Distribution Lab will not do Visualization
- What is deliberately wrong with dirty-customers Data quality
- Course note — Testing data like you test code Data quality
- Pattern: Counting your tests Experimentation
- The category that eats the chart Visualization
- A running total that does not lie about ties Analytics engineering
- Counting rows is not testing grain Analytics engineering
- unit_price_usd: mean 72.66, median 48.86 Statistics
- The top of temp_c is 1.1× the 99th percentile Statistics
- What humidity_pct actually contains in sensor-telemetry Data quality
- The overall average hides 7 different numbers Analytics practice
- r = -0.01 between distance_km and surge_multiplier Statistics
- Charting pm10 by city Visualization
- The chart rules behind Chart Builder Visualization
- Movie ratings, in one paragraph Data quality
- Reader question — What a data team is actually for Analytics practice
- Pattern: As-of feature computation Machine learning
- The category that eats the chart Visualization
- Comparing a period to the one before it Analytics engineering
- Is flight_no + carrier the grain of flight-delays? Analytics engineering
- tip_usd: mean 3.13, median 1.96 Statistics
- Five minutes with category in retail-orders Data quality
- Before you clip the outliers Statistics
April 30
- The overall average hides 2 different numbers Analytics practice
- The churn model that broke its own evaluation Machine learning
- r = 0.38 between tip_usd and surge_multiplier Statistics
- Charting temp_c by status Visualization
- Regression Lab works offline Visualization
- What is deliberately wrong with data-job-postings Data quality
- Reader question — Idempotency, backfills, and yesterday arriving twice Data engineering
- The category that eats the chart Visualization
- Deterministic surrogate key, in short Data engineering
- What flight_no actually contains in flight-delays Data quality
- Is posting_id + seniority the grain of data-job-postings? Analytics engineering
- Finding the gaps in city-air-quality Analytics engineering
- co2_tonnes_per_capita: mean 3.67, median 2.80 Statistics
- The top of dep_delay_min is 3.1× the 99th percentile Statistics
- The overall average hides 5 different numbers Analytics practice
- r = -0.09 between solar_mw and wind_mw Statistics
- Charting vibration_mm_s by site Visualization
- What Chart Builder will not do Visualization
- An exercise on ride-hail-trips Data quality
- Course note — What a data team is actually for Analytics practice
- A proportion with an interval, in short Statistics
- A running total that does not lie about ties Analytics engineering
- Counting rows is not testing grain Analytics engineering
- The category that eats the chart Visualization
- The overall average hides 14 different numbers Analytics practice
- Five minutes with priority in support-tickets Data quality
- The average revenue usd is not the typical one Statistics
- The top of pm25 is 2.7× the 99th percentile Statistics
- What cancelled actually contains in flight-delays Data quality
- r = 0.92 between distance_km and duration_min Statistics
March 31
- Charting temp_c by site Visualization
- Reading subject_id before trusting it in clinical-trial Data quality
- An exercise on city-air-quality Data quality
- Reader question — Causal claims when you cannot randomise Experimentation
- Reading temp_c before trusting it in grid-energy-load Data quality
- Pattern: A proportion with an interval Statistics
- We cut the modelling module in half Machine learning
- sensor-telemetry.status: 3 values, 100.0% in the top three Visualization
- What flight_date actually contains in flight-delays Data quality
- Sessions out of raw events in saas-subscriptions Analytics engineering
- Counting rows is not testing grain Analytics engineering
- Five minutes with country in world-indicators Data quality
- Before you clip the outliers Statistics
- seats_now: mean 59, median 8 Statistics
- What rated_at actually contains in movie-ratings Data quality
- The overall average hides 14 different numbers Analytics practice
- r = -0.37 between wind_mw and price_eur_mwh Statistics
- Charting seats by plan Visualization
- Reading release_year before trusting it in movie-ratings Data quality
- Retail orders, in one paragraph Data quality
- Reader question — Writing SQL people can read, and reading the plan Analytics engineering
- What dep_delay_min actually contains in flight-delays Data quality
- Anti-join, in short Analytics engineering
- The category that eats the chart Visualization
- Keeping one row per key in support-tickets Analytics engineering
- Counting rows is not testing grain Analytics engineering
- Five minutes with wind_mw in grid-energy-load Data quality
- A strong correlation, and what it is not Statistics
- Before you clip the outliers Statistics
- Five minutes with first_response_min in support-tickets Data quality
- The overall average hides 8 different numbers Analytics practice
February 28
- Charting fare_usd by borough Visualization
- Reading sessions before trusting it in ab-test-checkout Data quality
- Randomised trial results, in one paragraph Data quality
- Course note — Frame the problem, then beat a stupid baseline Machine learning
- What units actually contains in retail-orders Data quality
- ride-hail-trips.payment_type: 3 values, 100.0% in the top three Visualization
- Counting rows is not testing grain Analytics engineering
- Five minutes with rpm in sensor-telemetry Data quality
- Counting your tests, in short Experimentation
- Counting rows is not testing grain Analytics engineering
- CUPED gave us back 40% of our sample size for ten lines of code Experimentation
- What distance_km actually contains in ride-hail-trips Data quality
- Keeping one row per key in ab-test-checkout Analytics engineering
- Before you clip the outliers Statistics
- What line_no actually contains in retail-orders Data quality
- seats_now by industry: a 57% spread Analytics practice
- A essentially none correlation, and what it is not Statistics
- Charting bmi by arm Visualization
- Five minutes with discount_pct in retail-orders Data quality
- An exercise on grid-energy-load Data quality
- Cutting a lesson down — Schema drift, and the contract that prevents it Data engineering
- support-tickets.channel: 4 values, 87.8% in the top three Visualization
- Five minutes with returned in retail-orders Data quality
- age by sex: a 1% spread Analytics practice
- Pattern: Forward-chaining split Machine learning
- Finding the gaps in support-tickets Analytics engineering
- Before you clip the outliers Statistics
- What baseline_score actually contains in clinical-trial Data quality
January 31
- A essentially none correlation, and what it is not Statistics
- Charting population by country Visualization
- What returned actually contains in retail-orders Data quality
- What is deliberately wrong with movie-ratings Data quality
- Reader question — Joins that do not fan out Analytics engineering
- What life_expectancy actually contains in world-indicators Data quality
- Pattern: Contract check at the boundary Data quality
- Sessions out of raw events in support-tickets Analytics engineering
- Reading carrier before trusting it in flight-delays Data quality
- The top of age is 1.1× the 99th percentile Statistics
- Most of our experiments could never have worked Experimentation
- The category that eats the chart Visualization
- Five minutes with signed_up_on in saas-subscriptions Data quality
- Counting rows is not testing grain Analytics engineering
- The overall average hides 2 different numbers Analytics practice
- A weak correlation, and what it is not Statistics
- Charting mrr_usd by plan Visualization
- What is deliberately wrong with world-indicators Data quality
- Reading remote before trusting it in data-job-postings Data quality
- Course note — Communicating a result to someone who will not read your notebook Visualization
- Finding the gaps in sensor-telemetry Analytics engineering
- Five minutes with arm in clinical-trial Data quality
- Pattern: Gaps and islands Analytics engineering
- flight-delays.origin: 14 values, 22.6% in the top three Visualization
- What channel actually contains in support-tickets Data quality
- Counting rows is not testing grain Analytics engineering
- Before you clip the outliers Statistics
- What is_returning actually contains in ab-test-checkout Data quality
- salary_min_usd by country: a 379% spread Analytics practice
- A moderate correlation, and what it is not Statistics
- Charting vibration_mm_s by status Visualization