8 RAG Architectures: How They Work and When to Use Them

TL;DR

RAG means retrieval-augmented generation: finding relevant source material and giving it to a language model before it writes an answer. A language model generates text; retrieval supplies information it should use.

• These eight patterns change what we retrieve, how we search, or who decides the next step.
• They can overlap: an agent might use hybrid search over text and images, then check the results.
• More steps bring more cost and failure points. Choose a pattern for an observed need and verify its answers against real evidence.

This is a practical map, not an exhaustive taxonomy or a ladder from bad to good. We will use the same fictional dinner-policy question throughout. For document preparation and retrieval basics, start with how RAG works.

1. Naive RAG: retrieve, then answer

Emily asks whether she can claim a $45 dinner on a work trip. The simplest pipeline searches the policy collection once, puts matching passages beside her question, and asks the model to answer. This retrieve-then-generate baseline is commonly called Naive RAG. “Naive” describes its simple structure, not its usefulness. The original RAG paper connects retrieved documents with generation.

Before questions arrive, documents become a searchable index, often split into passages called chunks. One search option uses embeddings, numerical representations that help compare meanings. RAG can also use keyword search; a vector database is not mandatory.

This is a useful baseline for focused questions and well-organized sources. Its weak spot is straightforward: one missed passage can produce an incomplete answer. Retrieving the $50 limit without the receipt requirement loses a condition, even if the final prose sounds convincing.

2. Multimodal RAG: evidence beyond text

Suppose Emily also uploads a receipt image. Multimodal RAG retrieves and uses evidence in more than one form, such as text and images. The policy explains the requirement; the receipt supplies details to inspect. MuRAG is a research example that retrieves and reasons over images and text.

There are different implementations. Optical character recognition (OCR) extracts written text from an image so a text-based index can search it. Captions offer another text representation. Multimodal models can instead represent or inspect visual content alongside text. The right choice depends on whether words alone preserve the evidence or whether layout, diagrams and visual relationships matter.

Keep the original image and its location attached to any extracted text. A blurry total, missing page or misread line item should remain uncertain. Reading a receipt does not establish its authenticity, satisfy every policy condition, or authorize reimbursement.

3. HyDE: search with a hypothetical document

A short question may look very different from the documents that answer it. HyDE means Hypothetical Document Embeddings. A model first writes a plausible document in response to the question. The system embeds that generated text and uses it to find similar, real documents in the collection. The HyDE paper describes this generate, encode and retrieve sequence.

For Emily, the hypothetical passage might describe dinner expenses and receipt rules. Its vocabulary can help connect a casual question with formal policy language. The final answer should then use the actual Travel Policy v3 passage, not the generated draft.

This is worth testing when question wording and document wording are poorly matched. The risk is that an invented detail steers retrieval toward the wrong material. If the hypothetical document imagines a missing-receipt exception, that exception remains unsupported. HyDE adds a model call and needs comparison against directly searching the original question.

4. Corrective RAG: check the retrieved material

Corrective RAG asks whether retrieved material is useful enough before relying on it. In the original CRAG research, a retrieval evaluator assesses the results and selects actions: refine retrieved information, replace unsuitable results with web evidence, or combine both when uncertain. CRAG makes retrieval quality part of the control flow.

Emily’s search might return a general travel introduction without the dinner section. A check can flag that the required evidence is missing. A production design could then search another approved internal source or stop with an explicit limitation. These are related corrective choices, not a claim to reproduce the paper’s full algorithm.

Web fallback is appropriate only when authorized and relevant. Public pages cannot establish this fictional employer’s private policy, and private queries should not leak to external search. The evaluator can also be wrong: a high relevance score is not a truth certificate. Limit retries and retain a clear “insufficient evidence” outcome.

5. Graph RAG: follow relationships

A graph represents things as nodes and relationships as links between them. Graph-assisted RAG can retrieve through those relationships: Travel Policy v3 contains §4, and that section specifies both a dinner limit and a receipt requirement. Each relationship should remain traceable to source material.

This becomes useful when questions connect multiple documents or ask about themes across a collection. Microsoft’s GraphRAG is a specific approach: it extracts entities and relationships, groups related entities into communities, and prepares summaries that help answer broad questions. It is not the definition of every RAG system that uses a graph. The GraphRAG paper focuses on global questions over document collections.

Emily’s single dinner question may need only one passage. A graph becomes more relevant when a user needs connections across many policies. Extracting those links, updating them and maintaining permissions adds work. An incorrectly extracted relationship can spread an error; a summary should not hide the original evidence or its version.

6. Hybrid RAG: combine search methods

Hybrid retrieval commonly combines keyword search with dense-vector search, which compares embeddings. A keyword ranking method such as BM25 helps match exact terms; vector search can find passages with related meaning even when the wording differs. The system merges their candidates and may rerank them, meaning reorder them for relevance. Microsoft documents this combination of full-text and vector search.

For Emily, exact matching can preserve “Travel Policy v3” while semantic search connects “claim a dinner” to “dinner reimbursement.” Neither search alone necessarily captures both needs. Their results still need deduplication, version checks and permission filtering.

Combining graph and vector retrieval is also possible, but “hybrid” does not exclusively mean that pairing. Name the methods you actually use. Test their merged ranking against the separate baselines: adding another index does not automatically improve the evidence that reaches the model.

7. Adaptive RAG: choose the retrieval effort

Not every question needs the same amount of searching. Adaptive RAG chooses a route based on the question. The original Adaptive-RAG research uses a trained classifier to select no retrieval, one retrieval step, or iterative retrieval for more complex questions. Adaptive-RAG studies routing by question complexity.

A greeting may need no company sources. Emily’s dinner question needs the current policy. A question spanning several policies may need multiple searches. This design tries to spend retrieval effort where it helps, rather than applying the longest workflow to every request.

The router becomes another component to evaluate. It may classify a short but sensitive policy question as easy and skip needed evidence. Product rules should require sources where the answer depends on private or changing facts. Measure routing mistakes as well as answer quality, response time and cost; routing itself also consumes resources.

8. Agentic RAG: let an agent guide retrieval

An agent is a system that can choose a next action using a model and tools. In Agentic RAG, it can select a search tool, inspect results, refine a query and search again. One agent is enough; a team of agents is optional. LangChain describes agentic retrieval with one or more retrieval tools.

The agent could locate Emily’s policy, notice that the retrieved passage lacks the receipt condition, and fetch the relevant section. This flexibility helps with questions whose next search depends on what was just found. It also makes latency, cost and behavior less predictable than a fixed sequence.

Define allowed tools, maximum steps, timeouts and a stopping rule. Treat retrieved instructions as untrusted content. MCP, the Model Context Protocol, can provide a standard connection to tools and data, but is optional. A connection alone does not authorize access or business actions. Our example uses read-only retrieval; searches never authorize a payment.

Choose for the problem you have

Start with representative questions and a simple baseline. Inspect the failed examples before choosing another architecture. This table is a design guide, not a ranking or a benchmark.

For the dinner question, one authorized passage containing both conditions may be enough. Hybrid search might help find it; multimodal handling matters if an image supplies needed evidence. Add a graph or a planning loop only when your questions justify their extra maintenance and runtime work.

Check the evidence and the answer

Every pattern shares a basic obligation: enforce the user’s document access before assembling model context. Adding a graph, summary or second search index must not bypass those permissions. Document-level access control belongs in the retrieval design.

• Retrieval: did we find the current §4, including both conditions?
• Evidence: can each claim be traced to an accessible, intact source?
• Answer: does it keep the $50 limit and itemized receipt requirement?
• Missing information: does it avoid inventing a lost-receipt exception?
• Operation: are search steps, failures, response time and cost bounded and observable?

Test outdated policies, missing pages, misleading retrieved instructions and forbidden documents. Record source versions and retrieval decisions without exposing private content in logs. A citation, a model’s confidence or a favorable model-based grade is a signal to inspect, not proof that the answer is correct.

Continue with AI guardrails for access and untrusted-content boundaries, and AI evaluations for comparing changes against reviewed expectations. The architecture earns its place when it improves the task under those checks.

Frequently asked questions

Sources & further reading

The primary research behind the article, for readers who want to explore the details.

1. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks Lewis et al. · 2020
2. MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text Chen et al. · 2022
3. Precise Zero-Shot Dense Retrieval without Relevance Labels Gao et al. · 2023
4. Corrective Retrieval Augmented Generation Yan et al. · 2024
5. From Local to Global: A Graph RAG Approach to Query-Focused Summarization Edge et al. · 2024
6. Hybrid search using vectors and full-text search in Azure AI Search Microsoft Learn
7. Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity Jeong et al. · 2024
8. Retrieval LangChain Docs
9. Document-level access control in Azure AI Search Microsoft Learn