The data interview questions that actually predict performance

I have run somewhere north of two hundred interviews for data roles and made a fair number of bad hires along the way. The pattern in the bad ones is consistent: they passed a technical screen that tested recall, and failed at the parts of the job that are about judgement.

Here is what I would ask now, and what each question is for.

The ones that work#

"Here is a number that looks wrong. Walk me through what you would check."#

Give them something concrete: conversion dropped 18% on Tuesday. Then be a real stakeholder — answer their questions, and have an answer ready that they only find if they ask the right thing.

What you learn: whether they check the pipeline before the analysis, whether they ask about deploys and tracking changes, whether they think about denominators, whether they consider that the metric definition changed rather than the world. This single question separates people faster than anything else I have used.

The best answers start with "is the data complete?" rather than "let me segment it".

"Explain a technical concept to me as though I am a stakeholder."#

Pick something they claimed on their CV. A p-value, a join, a confidence interval, why the dashboard is slow.

What you learn: whether they understand it, and whether they can communicate. Those are different, and the job needs both. Most of the work in this field is explaining, and people who can only explain to other data people are limited in a way that shows up quickly.

"Tell me about an analysis you did that turned out to be wrong."#

Everyone has one. The people who claim not to are either inexperienced or not paying attention.

What you learn: honesty, and whether they built a habit from it. A good answer has a specific mechanism ("I averaged an average", "my join fanned out and I did not check row counts") and a specific change ("I now check row counts before and after every join"). A vague answer about "a stakeholder misunderstanding" is a bad sign.

"What would you need to know before starting this piece of work?"#

Give them a real, vague request: "leadership wants to know if our enterprise customers are healthy".

What you learn: whether they scope. Do they ask what decision it feeds, what "healthy" means, what the comparison is, when it is needed? This is the highest-leverage skill in the job and almost nobody tests for it.

One SQL question, with a real trap in it#

Not fifteen. One, with a deliberate grain problem — a table where the obvious query double-counts.

What you learn: whether they check their work. The candidates who ask "what is the grain of this table?" before writing anything are the ones who will not ship a wrong number, and it is a small population.

"How would you know if this stopped working?"#

After they have designed anything — a pipeline, a model, a dashboard.

What you learn: whether they think past the happy path. The people who mention monitoring, tests, and who gets paged are the people you can hand something to and stop worrying about.

The ones that do not work#

Whiteboard SQL from memory. Nobody writes SQL without a schema in front of them and autocomplete. You are testing memorisation and nerve.

Statistics trivia. "Define heteroscedasticity." Either they remember the word or they do not; neither tells you whether they would notice the fan shape in a residual plot.

Brain teasers. Well-studied, well-refuted, still in use.

Take-homes over four hours. They select for people with free time, which correlates with things you should not be selecting on. If you use one, cap it at two hours, say so honestly, and actually stick to it.

Long algorithm questions. For a data engineering role, some coding fluency is fair. Inverting a binary tree is not the job, and the industry mostly knows this and does it anyway.

What I would actually run#

Four stages, about four hours of candidate time total:

1. Screen (30 min). Motivation, background, one scoping question.
2. Technical (60 min). One SQL question with a grain trap, one debugging scenario, done together with a schema visible and search allowed. Watch how they work, not what they recall.
3. Case (60 min). A real, vague business question. Assess the scoping, the approach, and the communication. The answer matters less than the route.
4. Team (45 min). Past work, the wrong-analysis question, and their questions for you.

And a rule I would enforce: write down what you are testing for at each stage before you run it. Most interview loops test the same thing four times and miss two things entirely, and nobody notices because there is no written scorecard to compare against.

The thing nobody tests and should#

Whether the candidate will tell you something you do not want to hear.

The most valuable person on a data team is the one who says "this analysis does not support that conclusion" in a room where everyone wants it to. There is no clean way to test for it, but you can get at it: ask about a time they disagreed with a senior stakeholder, and listen for whether they escalated, caved, or found a third path.

That trait predicts more of the value of a data hire than any technical skill on the list.