Explainability

Explainability is the extent to which relevant people can receive a useful account of how or why a system produced a result.

Plan for this page

Understand one concept in plain language and connect it to nearby ideas.

You will leave with: An explanation may describe influential inputs, rules, evidence, uncertainty, process, or a simpler approximation of model behavior. Different audiences need different explanations, and a plausible explanation must itself be tested for faithfulness and usefulness.
Time: 3–8 min
Before you begin: No prerequisite. Keep one real example in mind.
Do this now: Read the explanation, then compare the example with the common misunderstanding.

Prepared next stepCompare explanation and evidence

ExplainabilityExplainability is the extent to which relevant people can receive a useful account of how or why a system produced a result.

Model
Evaluation
Transparency

Plain explanation

An explanation may describe influential inputs, rules, evidence, uncertainty, process, or a simpler approximation of model behavior. Different audiences need different explanations, and a plausible explanation must itself be tested for faithfulness and usefulness.

Example

A credit decision provides the main factors that lowered an application’s score, the data used, and a route to correct errors or appeal.

Why it matters

Explanations can support debugging, informed use, contestability, and accountability. They do not automatically prove that a decision is accurate, fair, or lawful.

Common misunderstanding

Misconception: a fluent model-generated rationale reveals the exact internal cause. Correction: a rationale can be a post-hoc story unless its relationship to the decision is validated.

Related concepts

Follow relationships, not only the alphabet

ModelA model is a learned or designed representation that maps inputs to outputs for a defined task or family of tasks.EvaluationEvaluation is the structured measurement and investigation of how a model or system performs against defined questions, criteria, and contexts.TransparencyTransparency is making relevant information about a system, its use, evidence, limits, and governance available to the people who need it.

Compare explanation and evidence

Sources and scope

What supports this object

Artificial Intelligence Risk Management Framework (AI RMF 1.0)U.S. National Institute of Standards and Technology

Includes explainability and interpretability among trustworthy-AI characteristics; the needed explanation depends on audience, purpose and risk.

Open source

Loading page content...

…