Benchmark

A benchmark is a standardized task, dataset, procedure, and metric used to compare model or system performance.

Plan for this page

Understand one concept in plain language and connect it to nearby ideas.

You will leave with: Benchmarks make repeated comparison easier, but they represent only selected capabilities and conditions. Results can change with model version, prompting, tools, data contamination, scoring, or adaptation to the test.
Time: 3–8 min
Before you begin: No prerequisite. Keep one real example in mind.
Do this now: Read the explanation, then compare the example with the common misunderstanding.

Prepared next stepPractice reading evaluation claims

BenchmarkA benchmark is a standardized task, dataset, procedure, and metric used to compare model or system performance.

Model
Dataset
Evaluation

Plain explanation

Benchmarks make repeated comparison easier, but they represent only selected capabilities and conditions. Results can change with model version, prompting, tools, data contamination, scoring, or adaptation to the test.

Example

Several vision models are compared on the same held-out image set using the same top-one accuracy calculation.

Why it matters

Benchmark headlines can guide research but do not automatically predict product usefulness or social impact. Read the task definition and limitations before generalizing.

Common misunderstanding

Misconception: leading one benchmark means a model is best for every purpose. Correction: rankings are scoped to a particular test and setup.

Related concepts

Follow relationships, not only the alphabet

ModelA model is a learned or designed representation that maps inputs to outputs for a defined task or family of tasks.DatasetA dataset is an organized collection of examples or records used for training, evaluation, analysis, or system operation.EvaluationEvaluation is the structured measurement and investigation of how a model or system performs against defined questions, criteria, and contexts.

Practice reading evaluation claims

Sources and scope

What supports this object

Stanford AI Index Report 2026Stanford Institute for Human-Centered AI

Synthesizes dated benchmark and capability evidence; individual results require the benchmark’s task, data, version, method and limitations.

Open source

Loading page content...

…