Retrieval-augmented generation concept

What is RAG?

Reference

What is RAG? RAG, short for retrieval-augmented generation, is a way for an AI system to find information in an external collection and give it to a language model as context for an answer. The collection may contain documents or other reference material. The system retrieves material in response to a question; the model then generates a response using the question and that material.

Reviewed 2026-10-09

A language model with an information source

RAG is a way to answer a question by retrieving relevant material from an external collection and giving it to a language model as context for its answer. The name expands to retrieval-augmented generation: retrieval finds information, and generation turns the question and that information into text.

The external collection may hold documents or other reference material, so the system can use information beyond what is stored in the model’s learned parameters. In the original research, the authors described RAG as combining a pre-trained language model’s parametric memory with non-parametric memory accessed through retrieval. RAG connects information retrieval with text generation.

From documents to an answer

Before answering, a system prepares its collection for search: it processes documents, divides them into smaller passages, and indexes them. Some systems turn passages into embeddings, which are numerical representations used for search. When a person asks a question, a retriever searches the index and selects passages that appear relevant.

The system adds selected passages to the question as context in a prompt, then sends that prompt to a language model to generate a response. This basic pattern has an indexing stage and a query-time stage. Search methods vary, and a system may rank retrieved passages before using them. RAG describes the connection between a searchable collection and generated answers, not just the search step.

How RAG relates to search and fine-tuning

A search engine returns documents or passages for a person to inspect; a RAG system uses retrieved material as context for generated text. The search component still matters, but the output also depends on how the language model uses the passages. RAG is useful when an application needs answers based on a chosen collection, such as a set of reference documents.

Fine-tuning is a neighbouring approach that adapts a model using additional training, while RAG fetches source material during question answering. These approaches can also be combined. RAG is not a synonym for a vector database: that database is one possible way to index material, while retrieval can use other search indexes. For more on the comparison, see RAG versus fine-tuning.

What retrieved evidence does and does not tell you

RAG can reduce unsupported guessing by giving a model relevant material, but it does not guarantee a correct answer. Check five things separately: retrieval quality (were useful passages found?), answer faithfulness (does the answer reflect them?), factual correctness (are its claims true?), citation support (does each cited source back the claim?), and permissions (was this user allowed to retrieve the material?).

A retrieved passage may itself be wrong, old, or incomplete, and an answer may not faithfully reflect a useful passage. A displayed citation also needs checking against the particular claim it accompanies. Permissions are a separate failure point: systems need to control which documents each user can retrieve. Microsoft’s guidance warns that missing access controls can expose sensitive content, and that retrieved material does not prevent inaccurate answers.

Questions people ask

What is retrieval-augmented generation, in plain words?

It is a way to generate an answer using both a language model and information retrieved from an external collection. The system finds passages related to a question, gives them to the model as context, and generates a response.

Does RAG require a vector database?

No. A vector database is one option. RAG can use different search indexes, including indexes based on word matching or dense vector representations.

How is RAG different from fine-tuning?

RAG retrieves reference material when someone asks a question. Fine-tuning adapts a model through additional training. A system can combine the two approaches.

Does RAG guarantee accurate answers or correct citations?

No. Retrieval can return irrelevant or incomplete material, and a model can still produce an inaccurate answer. A citation must also be checked to see whether it supports the particular claim it accompanies.

Learn more

Understanding Retrieval Augmented Generation
A practical overview of preparing a document collection, retrieving context, and passing it to a language model.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
The original research paper describes combining a language model with external memory accessed through retrieval.
Retrieval augmented generation (RAG) and indexes
Explains retrieval setup and practical limits, including incomplete search results and access control.

What changed

Sources

Part of Retrieval-augmented generation: The retrieval and answer pipeline Prepare a searchable collection Find and test failure points