AI glossary

What is Retrieval-augmented generation?

Also called RAG ·

Retrieval-augmented generation (RAG) is a technique where an AI system first retrieves relevant passages from an external source, such as your documents or a search index, and gives them to a language model so its answer is grounded in that material.

How it works

The term comes from a 2020 paper by Lewis and colleagues at Facebook AI Research, which combined a retriever with a generator model. Today's typical pipeline has two stages:

  1. Indexing (once, then kept up to date). Split documents into chunks, turn each chunk into an embedding, and store them in a vector database or search index.
  2. Answering (every query). Embed the user's question, retrieve the most relevant chunks (often with keyword search and a re-ranker as well), insert them into the prompt with instructions such as "answer only from these sources and cite them", and let the LLM write the answer.

Variants include agentic RAG, where an AI agent decides when and what to search, and graph-based RAG, which retrieves along links between entities.

Why it matters

  • Fresh and private knowledge. The model can answer about last week's policy or your internal wiki without retraining.
  • Fewer hallucinations, with citations. Grounding answers in retrieved text reduces hallucination and lets users check sources. It does not eliminate errors: bad retrieval still produces bad answers.
  • Cheaper than training. Updating an index is far cheaper than fine-tuning, and fine-tuning is better at teaching style or format than facts.
  • Access control. You can filter retrieval by user permissions, which is hard to do with knowledge baked into model weights.

Example

An HR assistant gets "How many days of parental leave do I get in Germany?" It retrieves the current German leave policy from the HR handbook, passes the two relevant paragraphs to the model, and answers with the figure and a link to the policy page.

Questions people ask

What is LLM vs RAG?+

An LLM is the model that generates text. RAG is a way of using an LLM: you retrieve relevant information first and put it in the prompt, so the LLM answers from that information instead of only from memory.

Is ChatGPT a RAG LLM?+

ChatGPT is not RAG by itself, but it uses retrieval when it searches the web or reads files you upload, which is a RAG pattern. Without those tools it answers from what the model learned in training.

Related terms

Sources

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv (Lewis et al., 2020)
  2. Lost in the Middle: How Language Models Use Long Contexts, arXiv (Liu et al., 2023)

Back to the AI glossary

Go deeper