Distribution shift

Distribution shift is a meaningful difference between the data or conditions used to develop a model and those encountered later.

Plan for this page

Understand one concept in plain language and connect it to nearby ideas.

You will leave with: Inputs, populations, measurement processes, behavior, environment, or the relationship between inputs and targets can change. A model may remain technically unchanged while its performance and consequences deteriorate.
Time: 3–8 min
Before you begin: No prerequisite. Keep one real example in mind.
Do this now: Read the explanation, then compare the example with the common misunderstanding.

Prepared next stepLearn why evidence needs context

Distribution shiftDistribution shift is a meaningful difference between the data or conditions used to develop a model and those encountered later.

Dataset
Generalization
Evaluation

Plain explanation

Inputs, populations, measurement processes, behavior, environment, or the relationship between inputs and targets can change. A model may remain technically unchanged while its performance and consequences deteriorate.

Example

A demand model trained before a major change in travel behavior may make poor forecasts afterward even if its software still runs correctly.

Why it matters

Deployment is not a one-time test. Monitoring inputs, outcomes, subgroups, and context helps reveal when past evidence no longer supports current reliance.

Common misunderstanding

Misconception: if a model passed evaluation once, its quality remains fixed. Correction: the world, users, data collection, and objectives can shift around it.

Related concepts

Follow relationships, not only the alphabet

DatasetA dataset is an organized collection of examples or records used for training, evaluation, analysis, or system operation.GeneralizationGeneralization is a model’s ability to perform usefully on relevant examples or conditions not seen during training.EvaluationEvaluation is the structured measurement and investigation of how a model or system performs against defined questions, criteria, and contexts.

Learn why evidence needs context

Sources and scope

What supports this object

Machine Learning GlossaryGoogle for Developers

Defines distribution shift and related changes between training and serving; real deployments need domain-specific monitoring.

Open source

Loading page content...

…