Evaluation

Evaluation is the structured measurement and investigation of how a model or system performs against defined questions, criteria, and contexts.

Plan for this page

Understand one concept in plain language and connect it to nearby ideas.

You will leave with: It can combine quantitative metrics, human review, scenario tests, subgroup analysis, red teaming, field observation, and monitoring. A useful evaluation states what was tested, on which data, under which conditions, and what remains unknown.
Time: 3–8 min
Before you begin: No prerequisite. Keep one real example in mind.
Do this now: Read the explanation, then compare the example with the common misunderstanding.

Prepared next stepLearn to evaluate errors and evidence

EvaluationEvaluation is the structured measurement and investigation of how a model or system performs against defined questions, criteria, and contexts.

Model
Generalization
Benchmark
Verification

Plain explanation

It can combine quantitative metrics, human review, scenario tests, subgroup analysis, red teaming, field observation, and monitoring. A useful evaluation states what was tested, on which data, under which conditions, and what remains unknown.

Example

Before deploying a support assistant, a team measures answer accuracy, unsupported claims, refusal errors, latency, privacy behavior, and escalation across realistic cases.

Why it matters

A demonstration or average score can conceal important failures. Evaluation connects a capability claim to evidence and the actual consequences of intended use.

Common misunderstanding

Misconception: one accuracy number completely describes a system. Correction: relevant quality includes error types, contexts, groups, uncertainty, safety, operations, and human outcomes.

Related concepts

Follow relationships, not only the alphabet

ModelA model is a learned or designed representation that maps inputs to outputs for a defined task or family of tasks.GeneralizationGeneralization is a model’s ability to perform usefully on relevant examples or conditions not seen during training.BenchmarkA benchmark is a standardized task, dataset, procedure, and metric used to compare model or system performance.VerificationVerification is the process of checking whether a claim, output, action, identity, or result is adequately supported for its intended use.

Learn to evaluate errors and evidence

Sources and scope

What supports this object

Artificial Intelligence Risk Management Framework (AI RMF 1.0)U.S. National Institute of Standards and Technology

Frames measurement and management of validity, reliability and risk in context; it does not prescribe one metric for every system.

Open source

Loading page content...

…