Skip to the content
AI Explainedpreview Learning Center

AI Explained

Giving a model your own facts

Retrieval augmented generation, or RAG: look the answer up first, then let the model write it.

The pipeline

Look it up, then answer

A model only knows what it was trained on and what you put in front of it. Retrieval puts the right pages in front of it.

searchYour documentsPolicies, manuals,ticketsPassagesCut into small piecesThe indexEach passage stored bymeaningA questionAsked by a personRetrieveFind the closestpassagesThe promptQuestion plus passagesThe modelAnswers from what itwas given

Press Play to walk through it, or Listen to have it read to you.

Read all the steps
  1. Ahead of time, your documents are cut into short passages.
  2. Each passage is stored in an index that can be searched by meaning, not only by exact words.
  3. Later, someone asks a question.
  4. The question is used to search the index for the passages closest to it.
  5. Those passages are placed in the prompt, next to the question.
  6. The model answers from what is in front of it, so it can use facts it was never trained on and point to where they came from.

The idea

What it is and why it exists

What it is

Retrieval augmented generation: relevant information is fetched from an outside knowledge base when the question is asked and passed into the context window with it, so the answer is grounded in that information [1].

Why it was invented

A pretrained model stores what it knows in its parameters, and its ability to reach and use that knowledge precisely is limited. Showing where an answer came from and updating what the model knows were open problems [2].

Two memories

The paper that named it combines the model's own trained memory with a second memory outside it: an index of documents that a retriever searches [2].

What it gives you

Answers from your own documents, a source to show for each one, and knowledge that changes the moment the documents do, with no retraining.

Try it

Ask the handbook

Six lines from an invented staff handbook are the whole knowledge base. Ask a question and see which lines are retrieved and the prompt the model would get.

This demonstration matches words. Real systems compare meaning, so a question can find a passage that shares no words with it. The handbook is invented.

In practice

Where it goes wrong

Bad search, bad answer

The model can only use what the search brings back. Most problems with these systems are search problems.

Stale and duplicate documents

Two versions of a policy give two answers. Clean the collection before indexing it.

Access rules

If the index ignores who may read what, it becomes a search engine for confidential files.

Hidden instructions

A document can carry text written to steer the model. Anyone who can add a document can try.

Sources

Where this comes from

Read 2026-11-15, unless a source gives its own date. A number in brackets points to the page a statement was read on. Sentences with no number, the small demonstrations and the tools are this center's own explanations.

  1. Claude Platform Docs: Glossary
  2. Lewis and others: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)