Article · May 30, 2024
Dimensional modelling is not obsolete
Every few years someone announces that cheap compute has killed the star schema. It has not. Here is what actually changed, and what did not.
The argument goes: dimensional modelling was a response to expensive storage and slow joins. Storage is now cheap and joins are fast, so we can dump everything into one wide table and stop thinking about it.
Half of that is right. The conclusion is wrong, and the reason is that dimensional modelling was never primarily a performance technique.
What it was actually for#
Kimball's star schema optimises for a human being able to answer a question without asking anyone. The performance benefits were real and secondary. The primary benefit was cognitive: a fact table surrounded by dimensions is a shape a person can hold in their head, and the query for any question is one hop from the fact to whatever you want to slice by.
Cheap compute did not make that irrelevant. Cheap compute made it more relevant, because now everyone in the company can run queries, and most of them do not know what your tables mean.
What genuinely changed#
Normalisation for storage is over. Snowflaking a dimension into six tables to avoid repeating a category name was always a storage optimisation, and that optimisation is now pointless. Denormalise your dimensions flat.
ELT replaced ETL. Keeping raw data and transforming inside the warehouse is strictly better than transforming in flight. This changed where the modelling happens, not whether it should.
Wide tables became viable for specific jobs. A single denormalised table with 200 columns is genuinely good for a machine learning feature store or a BI tool that struggles with joins. This is a serving-layer decision, and it should be derived from a modelled layer rather than replacing it.
Semantic layers arrived. Metrics defined once, consumed everywhere. Complementary to dimensional modelling, not a substitute — a semantic layer needs well-shaped tables underneath or it is defining metrics over a swamp.
What did not change#
Grain still has to be declared. The most common source of wrong numbers is still someone not knowing what one row represents. One wide table with mixed grain is worse than a star schema, not better, because the mixing is now invisible.
Conformed dimensions still matter. If customer means something different in the marketing mart and the finance mart, no amount of compute makes those numbers reconcile. Agreeing on one customer dimension is an organisational act, and it is the highest-value thing an analytics engineer does.
Slowly changing dimensions still need a decision. "Was last February's revenue German or Spanish, given the customer moved in March?" is a question about your business, not about your warehouse. One Big Table does not answer it; it just makes the question harder to ask.
Additivity still has rules. You still cannot average an average. You still need to store components rather than ratios.
Where the wide-table crowd has a point#
They are right that a lot of modelling is over-engineered. Specifically:
- Building a full star schema for a dataset queried by three people is waste.
- Surrogate keys on dimensions nobody ever type-2s add ceremony for nothing.
- A junk dimension for four boolean flags is a solution to a problem that no longer exists.
- Bridge tables for many-to-many relationships are often worse than an array column in a modern warehouse.
The reasonable position: model the grain and the conformed dimensions rigorously, and be relaxed about everything else. Declare what one row is, agree on what a customer is, and stop there unless something forces you further.
What I actually build now#
Three layers, and the middle one is where the modelling lives:
Staging. One model per source table, renamed and typed, no joins, no logic.
Marts. Facts and dimensions, denormalised, grain declared in the description, uniqueness tested on that grain. Type-2 only on the dimensions people slice revenue by. Type-1 on everything else, without apology.
Serving. Whatever shape the consumer wants — a wide table for a feature store, a pre-aggregated rollup for a dashboard, a semantic layer for ad hoc questions. Derived, cheap to rebuild, and explicitly disposable.
That third layer is where the wide-table argument belongs. It is a serving decision. Making it your only layer means you have no place to put the grain, and the grain is the part that goes wrong.
The test#
Ask a new analyst to compute revenue by country for last quarter. Time it.
If they can do it in five minutes without asking anyone, your model is good, whatever its shape. If they need a Slack thread, three follow-up questions, and a warning about which of the two revenue columns to use, the model has failed — and it will keep failing, once per new analyst, forever.
That test does not care about your architecture. It cares whether the shape of your data matches the shape of the questions.