Generative models produce new sequences or media from learned patterns and current context. This explains fluency and variation without turning output into memory, intention, or verified knowledge.
Plan for this page
Understand the idea, test it actively, then explain it in your own words.
You will leave with
explain tokens, context, prediction, and sampling as a useful language-model abstraction recognize why plausible continuation can be useful and unsupported state the limits of the abstraction across models, modalities, and products
Time
35 min
Before you begin
AI from Zero is useful preparation, but it is not a locked prerequisite. Each unit states the concepts it uses.
Do this now
Read the outcomes and useful terms before the first section.
explain tokens, context, prediction, and sampling as a useful language-model abstraction
recognize why plausible continuation can be useful and unsupported
state the limits of the abstraction across models, modalities, and products
Terms you will use
These definitions prepare you for the reading; you do not need to memorize them.Token
A unit used by a model to represent and process a sequence; it may be a word part, character-like piece, code fragment, or other learned unit.
Context
The information available during the current generation, such as instructions, conversation, retrieved material, examples, and tool results.
Sampling
A procedure for selecting among possible continuations from a model's output distribution, influenced by settings and product rules.
01
Generation is structured continuation
A language model processes tokens and estimates possible continuations from patterns shaped during training and the current context. The product selects tokens repeatedly, producing a sequence. Other generative systems use related ideas over image, audio, video, or multimodal representations.
‘Predict the next token’ is a useful entry point, not a complete description of every internal representation, training objective, reasoning behavior, architecture, or product layer. Keep the model simple enough to reason with and explicit enough not to become a myth.
02
Fluency follows pattern; truth requires another relation
Training examples contain language patterns, factual associations, styles, errors, contradictions, and social conventions. A model can produce a coherent continuation that matches these patterns without checking the claim against a current primary source.
The same mechanism can summarize supplied text, generate an analogy, draft code, or invent a citation. Usefulness depends on task and controls. Verification asks a different question: what evidence supports this exact claim in this context now?
03
Variation is designed, not mysterious mood
Possible continuations have different model-assigned weights. Sampling settings, system instructions, tool results, safety rules, model versions, and product code affect which output appears. Repeating a prompt may therefore produce variation.
Variation can support brainstorming but complicates reproducibility and evaluation. For consequential use, record model and product versions, inputs, sources, settings where available, review steps, and the final human decision.
Interactive model
A simplified generation loop
Selected layerAssemble context
Instructions, user input, conversation, retrieved evidence, examples, memory, and tool output are selected and ordered.
Read the complete text alternative
Assemble contextInstructions, user input, conversation, retrieved evidence, examples, memory, and tool output are selected and ordered.
RepresentThe model transforms tokens or other inputs into internal representations shaped by its training.
Estimate and selectPossible continuations receive weights; decoding or sampling plus rules selects an output step.
Repeat and wrapThe loop continues, while the product displays, filters, cites, stores, routes, or acts on the result.
Misconception to correct
“A fluent answer is a retrieved paragraph from a clean database inside the model.”
Generated text is constructed from learned parameters and current context. Products may also retrieve documents, but generation and retrieval remain distinct operations.
Active checks
Decide, then compare the reasoning
No grade, score, or streak. Progress records completed learning objects, not your worth or ability.01
Why can a language model produce a convincing but invented book citation?
02
The same prompt gives two different drafts. What is the best explanation?
Explain back and transfer
Explain generation without magic
Write a short explanation connecting context, learned patterns, possible continuations, selection, and verification. Name one limit of your explanation.
Answer both checks before completing.
Source context
What supports this unit and where it stops
Evidence status: established
Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)U.S. National Institute of Standards and Technology
Institutional account of generative-AI risks including confabulation; it is not a beginner mechanism textbook or legal compliance guarantee.
Elements of AI: Introduction to AIUniversity of Helsinki and MinnaLearn
Conceptual foundations for probability, learning and neural networks; use as a substantial external course, not as a source for product-specific behavior.
inAi