Contract check at the boundary

You meet it when#

An upstream team ships a change on a Thursday and tells nobody, because they have no idea you read that column.

The pattern#

Check at the boundary, not at the point of consumption. One check protects every model downstream of it.

from dataclasses import dataclass

@dataclass(frozen=True)
class ColumnSpec:
    name: str
    dtype: str
    nullable: bool = False

def validate(df, contract) -> None:
    """Fail the run with a message an on-call engineer can act on."""
    problems = []
    for spec in contract:
        if spec.name not in df.columns:
            problems.append(f"missing column: {spec.name}")
            continue
        actual = str(df[spec.name].dtype)
        if not actual.startswith(spec.dtype[:3]):
            problems.append(f"{spec.name}: expected {spec.dtype}, got {actual}")
        if not spec.nullable and df[spec.name].isna().any():
            problems.append(f"{spec.name}: {int(df[spec.name].isna().sum())} nulls in a non-nullable column")

    extra = set(df.columns) - {s.name for s in contract}
    if extra:
        print(f"note: new upstream columns ignored: {sorted(extra)}")   # additive is fine

    if problems:
        raise ValueError("Contract violation:\n  " + "\n  ".join(problems))

The asymmetry that matters#

A missing column is fatal. A new column is a log line.

Additive change should never wake anyone up — upstream teams ship features constantly and none of it concerns you. A removal or a type change always should, because it means a model downstream is about to produce nulls or a silently coerced value.

Getting this backwards gives you either a pipeline that cries wolf or one that never speaks.

What it cannot catch#

Semantic drift. status = 'active' used to include trials and now does not. Same name, same type, non-null, contract satisfied, numbers wrong.

No structural check finds that. The defences are a semantics field in the contract (so there is something to point at) and distribution monitoring — watching whether the proportion of each category moved overnight. Track null rate, distinct count and category shares per load, and alert on the change rather than the level.

That is what would have caught the silent else branch: the share of rows falling into the default went from 0% to 4%, and no schema check can see that.