AI Explained
Giving a model your own facts
Retrieval augmented generation, or RAG: look the answer up first, then let the model write it.
The pipeline
Look it up, then answer
A model only knows what it was trained on and what you put in front of it. Retrieval puts the right pages in front of it.
Press Play to walk through it, or Listen to have it read to you.
Read all the steps
- Ahead of time, your documents are cut into short passages.
- Each passage is stored in an index that can be searched by meaning, not only by exact words.
- Later, someone asks a question.
- The question is used to search the index for the passages closest to it.
- Those passages are placed in the prompt, next to the question.
- The model answers from what is in front of it, so it can use facts it was never trained on and point to where they came from.
The idea
What it is and why it exists
What it is
Retrieval augmented generation: relevant information is fetched from an outside knowledge base when the question is asked and passed into the context window with it, so the answer is grounded in that information [1].
Why it was invented
A pretrained model stores what it knows in its parameters, and its ability to reach and use that knowledge precisely is limited. Showing where an answer came from and updating what the model knows were open problems [2].
Two memories
The paper that named it combines the model's own trained memory with a second memory outside it: an index of documents that a retriever searches [2].
What it gives you
Answers from your own documents, a source to show for each one, and knowledge that changes the moment the documents do, with no retraining.
Try it
Ask the handbook
Six lines from an invented staff handbook are the whole knowledge base. Ask a question and see which lines are retrieved and the prompt the model would get.
This demonstration matches words. Real systems compare meaning, so a question can find a passage that shares no words with it. The handbook is invented.
In practice
Where it goes wrong
Bad search, bad answer
The model can only use what the search brings back. Most problems with these systems are search problems.
Stale and duplicate documents
Two versions of a policy give two answers. Clean the collection before indexing it.
Access rules
If the index ignores who may read what, it becomes a search engine for confidential files.
Hidden instructions
A document can carry text written to steer the model. Anyone who can add a document can try.
Sources
Where this comes from
Read 2026-11-15, unless a source gives its own date. A number in brackets points to the page a statement was read on. Sentences with no number, the small demonstrations and the tools are this center's own explanations.