AI Explained
How a language model works
It predicts what comes next. That one step, repeated, writes essays, code and answers.
The loop
One token at a time
A language model does one thing: given some text, it works out what is likely to come next. Everything it writes is that step repeated. Play the diagram, then try each part yourself below.
Press Play to walk through it, or Listen to have it read to you.
Read all the steps
- Your text is cut into tokens: small pieces that can be words, parts of words or single characters.
- The model reads all the tokens together. Its attention mechanism lets every token look at every other one.
- It gives a score to every token it knows: how likely each one is to come next.
- One token is picked. A low temperature takes the likeliest, a high one takes more chances.
- That token is added to the answer.
- Then the whole thing runs again with the new token included, until the answer is finished.
The parts
Six ideas that explain most of it
A model that predicts what comes next
A large language model is an AI model with a very large number of parameters, trained on vast amounts of text [1]. In its first round of training, called pretraining, it learns to predict the next word [1].
Tokens, not words
A model reads and writes tokens: the smallest units it handles, which can be words, parts of words, characters or bytes [1]. A common rule of thumb for English is that one token is about four characters, or three quarters of a word [2]. Newer Claude models use a tokenizer where a token is about 3.5 characters [1], and other languages can need more tokens for the same text [2].
Attention
Today's models are built on the Transformer, a design based only on attention mechanisms, introduced in 2017 [3]. Attention is what lets the model weigh every token against every other one when it decides what comes next.
Temperature
A setting that controls how random the choice of the next token is: higher gives more varied text, lower gives more predictable text. Even at zero the results are not fully deterministic [1].
The context window
All the text the model can refer to when it writes a reply, including the reply itself. It is working memory, separate from what the model learned in training [4].
Try it
See your text as tokens
Type anything. The pieces below show the idea of tokens and how the count grows.
The splitting here is a simple illustration, not a real tokenizer: real ones are learned from data and differ from model to model. The estimate uses the four-characters rule of thumb.
Try it
Pick the next word
The bars are the model's scores for the next word. Move the temperature and watch them change, then press Pick a few times.
The words and scores here are made up for the demonstration. A real model scores tens of thousands of tokens at every step.
The context window
Everything the model can see at once
It fills as you talk
Every turn, the model is sent the whole conversation again plus your new message, and its reply becomes part of the next turn. The system prompt, documents, images, tool definitions and tool results all count [4].
What happens when it is full
If the input alone is larger than the window, the request is refused with an error saying the prompt is too long [4].
Bigger is not always better
The documentation warns of context rot: as the number of tokens grows, accuracy and recall get worse, so choosing what goes in matters as much as how much fits [4].
How big it is
It depends on the model. The Claude documentation lists a window of one million tokens for its current models and 200,000 for older ones [4].
A planning aid. The sizes you enter are your own estimates; the two window sizes are the ones the Claude documentation lists.
Sources
Where this comes from
Read 2026-11-15, unless a source gives its own date. A number in brackets points to the page a statement was read on. Sentences with no number, the small demonstrations and the tools are this center's own explanations.