Field note · November 26, 2024

Three dashboards, three revenue numbers, ninety minutes

Nobody's SQL was wrong. That was the problem.

1 min read ·Analytics engineering ·warehousing practice

Finance: 4.1M. Product dashboard: 4.4M. Sales spreadsheet: 3.9M.

We spent ninety minutes on it. Every query was correct. Every number was defensible.

The choices each person had silently made, for one metric:

  • Gross or net of discounts?
  • Before or after refunds, and refunds attributed to the original order date or the refund date?
  • Booked, invoiced, or recognised?
  • Which currency, at which date's exchange rate?
  • Internal test accounts in or out?
  • Orders placed, or orders shipped?
  • Which timezone defines a day?

Seven roughly-binary choices is 128 defensible numbers. Nobody was wrong. Nobody had written it down.

We now define metrics in one place, in code, with a field most definitions skip:

yaml
- name: net_revenue
  calculation: sum(revenue_usd) - sum(refund_usd)
  time_column: ordered_at
  description: >
    After discounts and refunds, attributed to the ORIGINAL order date.
    Excludes internal test accounts. Non-USD at the daily ECB rate on the
    order date. Restated for up to 7 days.
  not_this: >
    NOT recognised revenue — finance defers subscription amounts across
    the term. See `recognised_revenue`.

not_this is the field that earns its place. Saying what a metric is not is what stops it being used for the thing it superficially resembles, and that is the mistake that reaches board decks.

We have fifteen certified metrics. Everything else is explicitly labelled "somebody's query", which makes the ad hoc tier legitimate rather than shadow work. That legitimacy is what stops people routing around the process.