Retrieval-Augmented Generation for Large Language Models: A Survey (Gao et al. 2023)

1. How the survey maps RAG

Retrieval-Augmented Generation for Large Language Models: A Survey organizes an engineering field that was expanding quickly. Gao et al. report covering more than 100 studies, 26 task categories, and nearly 50 datasets. These figures describe the scope of the survey at the time of publication, not the current size of the RAG literature.

RAG allows a language model to consult external, updatable knowledge before or during generation. This design helps address the limits of purely parametric memory. Facts can become outdated, specialized knowledge may be sparse, and answers generated only from model weights offer little information about their sources. The foundational NeurIPS 2020 RAG paper combined a sequence generator with a dense Wikipedia index. Gao et al. expand that view from a single model design to a complete generative-engine architecture.

This mechanism underpins generative engine optimization. GEO depends on systems that discover, select, and use external sources, but an explanation of the mechanism does not establish that any optimization method works.

What the survey does	What the survey does not do	Why it matters for GEO
Organizes RAG paradigms, components, and evaluation methods	Reverse-engineer a production search system	Shows the stages a source must pass through
Synthesizes results from earlier research	Conduct a single controlled benchmark	Separates evidence about system mechanics from evidence about tactics
Provides shared vocabulary for system design	Test content rewrites or citation visibility	Helps prevent retrieval results from being mistaken for attribution results

2. How RAG moves from indexing to generation

The paper presents a basic three-stage pipeline. First, documents are divided into retrievable units and indexed. A query then retrieves a small set of candidates. Finally, the generator receives the query together with selected context and produces an answer.

documents -> [INDEX] -> searchable chunks
                              |
query ----> [RETRIEVE] -> candidate set (top-k chunks)
                              |
                     [SELECT / GROUND] -> selected context
                              |
                         [GENERATE] -> answer

RAG relies on three distinct forms of stored information. Parametric memory is encoded in the model’s weights. The external knowledge base is the searchable corpus. Query-time context is the small subset placed in the model’s working input. Updating the corpus can change later answers without changing the model weights, but only if the new material is discovered, retrieved, and selected.

Production systems rarely complete this process in a single pass. They may rewrite the query, search several times, rerank candidates, compress context, identify a gap in the evidence, and retrieve again. RAG describes the architecture, while the answer loop describes the sequence of operations at runtime.

Stage	Input	Decision	Downstream failure
Indexing	Source documents	Which units and metadata become searchable	Missing, stale, or incoherent chunks cannot become candidates
Retrieval	Query and index	Which chunks enter the candidate set	Low recall omits evidence, while low precision adds noise
Selection	Candidate chunks	Which evidence remains in the context	Reranking or compression removes a useful passage
Generation	Query and context	How the evidence is combined into an answer	The model ignores, distorts, or overstates the evidence

3. Three RAG paradigms: Naive, Advanced, and Modular

3.1 Naive RAG

Naive RAG is the paper’s three-step baseline: index, retrieve, and generate. The word “Naive” is an architectural label, not a judgment about the quality of every simple implementation. A well-curated corpus and a precise query can make a basic pipeline effective, while a larger and more complex pipeline can still fail.

Input	Processing	Output	Common failure
Documents	Fixed chunking and indexing	Searchable corpus	Chunk boundaries divide the evidence or remove its context
User query	One retrieval call	Top-k passages	Poor precision, poor recall, or conflicting candidates
Query plus passages	One generation call	Natural-language answer	Hallucination, irrelevant detail, bias, or evidence misuse

Gao et al. identify weaknesses in retrieval, generation, and augmentation. Retrieval may miss relevant passages or return distracting ones. The generator may ignore the evidence, repeat it without resolving conflicts, or rely on it too heavily. A single retrieval step may also be insufficient when a question reveals additional subproblems only after the system begins examining the evidence.

3.2 Advanced RAG

Advanced RAG improves both the inputs to retrieval and the results it produces. Before retrieval, the system makes the index and query more discriminating. After retrieval, it reduces noise before the generator receives the context. The goal is not to add more components, but to produce better candidates and cleaner context.

Location	Technique family	Failure it addresses
Pre-retrieval: corpus	Chunk strategy, metadata, hierarchical indexes	Evidence is split, context-free, or hard to filter
Pre-retrieval: query	Rewrite, expansion, decomposition, routing	User wording does not match the corpus
Pre-retrieval: representation	Embedding adaptation, sparse/dense fusion	Similarity search misses exact terms or semantic matches
Post-retrieval: ranking	Reranking and relevance filtering	The top-k ranking contains plausible but weak evidence
Post-retrieval: context	Deduplication and compression	The prompt is redundant, conflicting, or too long

These interventions introduce trade-offs of their own. Aggressive compression can remove important qualifications. Query expansion can improve recall while drifting away from the user’s intent. Rerankers can add another opaque layer of preference. “Advanced” therefore refers to the number of available control points, not to guaranteed quality.

3.3 Modular RAG

Modular RAG treats search, memory, routing, fusion, prediction, and task adaptation as components that can be combined in different ways. A workflow may be sequential, conditional, iterative, or adaptive. Based on its intermediate state, the system may choose a source, issue a follow-up query, merge results, or stop.

Paradigm	Flow	Retrieval timing	Post-processing	Model adaptation	Typical limit
Naive	Linear	Once, before generation	Minimal	Usually none	Brittle when one retrieval misses
Advanced	Enhanced linear	Usually once, with a transformed query	Rerank, filter, compress	Optional	More stages, tuning, and latency
Modular	Composed and conditional	Once, iterative, recursive, or adaptive	Module-dependent	Prompting, adapters, or joint training	Complex control logic and hard-to-localize errors

This paradigm anticipated later agentic and deep-research systems, although the 2024 survey did not fully describe today’s agentic taxonomy. A later Agentic RAG survey adds reflection, planning, tool use, multi-agent coordination, and explicit control structures. This later work extends the modular approach. It does not show that Gao et al. had already covered these developments.

4. Three technical axes: retrieval, generation, and augmentation

4.1 Retrieval: what enters the candidate set

The survey examines retrieval at three levels: the corpus, the query, and the way each is represented. Chunk size and metadata determine what the system can match. Query rewriting and routing determine what it searches for. Sparse retrieval preserves lexical matches, while dense retrieval looks for semantic similarity. Hybrid approaches try to retain both kinds of signal.

Lever	Intended benefit	Failure when poorly configured
Chunk size and overlap	Preserve answer-sized evidence with enough context	Tiny fragments lose meaning, while large chunks dilute relevance
Metadata and hierarchy	Filter by source, time, section, or entity	Missing or wrong metadata hides good evidence
Query rewriting	Align user language with corpus language	Rewrite drifts from the original intent
Sparse/dense/hybrid choice	Balance exact-term precision and semantic recall	One representation suppresses the other signal
Retriever alignment	Adapt ranking to the task and generator	Better offline relevance scores fail to improve answers

Retrieval recall is necessary, but it is not sufficient. A relevant chunk in the top-k results can still be discarded, truncated, contradicted, or ignored. Conversely, a correct answer may come from the model’s memory even when retrieval fails. End-to-end accuracy alone cannot distinguish between these outcomes.

4.2 Generation: what survives and how the model uses it

Selection after retrieval determines which candidates enter the prompt. The way the generator is adapted then affects whether the model follows the evidence, combines several passages, declines to answer unsupported questions, or falls back on parametric memory.

These states are distinct. Retrieval does not guarantee selection, selection does not guarantee use, and use does not guarantee credit. Gao et al. explain retrieval and selection well. Whether the product credits the evidence depends on its own rendering and source policies, so citations and mentions must be measured separately.

State	What happened	Observable result
Retrieved	The source entered a candidate set	Usually hidden unless the system is controlled
Selected	Some source text entered the model’s context	Visible in traces, but not necessarily in the answer
Used	The answer depends on the evidence	Detectable through attribution or counterfactual analysis
Credited	The interface names or links the source	A visible link, citation marker, or mention

Keeping these states separate prevents a common overstatement: a grounded answer is not necessarily a well-attributed answer. A system can use evidence faithfully without naming its publisher. It can also cite a page that contains the claim without showing that the cited passage actually influenced the answer.

4.3 Augmentation: what, when, and how to retrieve

The augmentation axis asks three questions: what knowledge should be added, at what point in the model lifecycle it should be introduced, and how the system should repeat or adapt retrieval. These are independent design choices, not stages in a single maturity model.

Axis	Options in the survey	Design question
Stage	Pre-training, fine-tuning, inference	At which point should external knowledge affect behavior?
Source	Unstructured text, structured data, model-generated material	Which form of knowledge fits the task and its trust requirements?
Process	Once, iterative, recursive, adaptive	Does one retrieval suffice, or should evidence change the next query?

These distinctions rule out two oversimplifications. RAG is not synonymous with vector search, because the source can be structured and the retriever can use lexical or hybrid methods. Nor is RAG necessarily limited to inference. Retrieval can support training or adaptation, and the survey treats RAG and fine-tuning as potentially complementary.

5. Evaluating retrieval and generation separately

5.1 Evaluation targets and required abilities

The survey evaluates retrieval quality separately from generation quality. For retrieval, context relevance and recall indicate whether the system found useful evidence. For generation, faithfulness and answer relevance indicate whether the output is supported by that evidence and responds to the question.

Target	Required ability	Representative measures
Retrieval	Find useful evidence without overwhelming the prompt	Context relevance, precision, recall
Generation	Stay supported by the supplied evidence	Faithfulness or groundedness
End-to-end answer	Address the user’s actual question	Answer relevance and task accuracy
Robustness	Resist irrelevant or misleading context	Noise robustness
Abstention	Reject questions unsupported by the corpus	Negative rejection
Synthesis	Combine evidence across passages	Information integration
Conflict handling	Avoid blindly adopting false supplied claims	Counterfactual robustness

Evaluating these abilities individually reveals more about RAG failures than a single QA score. The paper also acknowledges that RAG-specific evaluation was neither mature nor standardized. A later evaluation survey formalized a unified process based on relevance, accuracy, and faithfulness, while again identifying limitations in the available metrics and benchmarks.

5.2 Benchmarks and tools in the paper

Gao et al. use representative benchmarks and evaluation frameworks to describe the state of the field. The list reflects the paper’s March 2024 snapshot and is not a current recommendation of evaluation tools.

Benchmark or tool	Main target in the survey	Role in the paper’s snapshot
RGB	Noise, rejection, integration, and counterfactual robustness	Tests how retrieved context changes behavior
RECALL	Counterfactual and knowledge-conflict behavior	Probes reliance on retrieved versus parametric knowledge
CRUD	Create, read, update, and delete tasks	Broadens evaluation beyond question answering
RAGAS	Retrieval and generation metrics	Reference-free or model-assisted component scoring
ARES	Context relevance, faithfulness, answer relevance	Automated evaluation with trained judges
TruLens	Instrumentation and feedback functions	Operational inspection of RAG applications

The value of these tools lies in diagnosis, not in choosing any particular tool. When a score falls, practitioners need to know whether the cause lies in the corpus, retriever, selector, generator, or evaluator. A single composite score can hide a good answer produced despite failed retrieval, or a fluent answer based on poor evidence.

5.3 What the evaluation layer still misses for GEO

RAG evaluation asks whether a system found useful context and produced a relevant, faithful answer. GEO requires an additional publisher-level view: whether a page was retrieved, used, named, linked, or cited prominently. The two groups of outcomes overlap, but they are not interchangeable.

System-quality outcome	Publisher-visible outcome
The system found relevant context	A specific page appeared among the candidates or cited sources
The answer is faithful to context	The cited passage actually supports the attributed claim
The answer addresses the query	The publisher received a link, citation, mention, or no credit
The system resists noisy evidence	A low-quality competitor did not displace the publisher’s source

This difference explains why improvements on a RAG benchmark cannot be converted directly into claims about citation share. The unit of evaluation changes from the correctness of the system to the exposure and attribution of an individual source.

6. Critical assessment: durable taxonomy, dated evidence

6.1 What the survey gets right

The survey’s lasting contribution is its organization of a large body of research. It gives researchers and practitioners shared language for systems that can differ sharply in their implementation.

Contribution	Why it remains durable	Present-day caveat
Naive → Advanced → Modular map	Clarifies increasing levels of control and composition	The categories overlap and are not strict maturity levels
Retrieval / generation / augmentation axes	Shows why RAG is more than “vector search”	Security and multimodality now need more explicit treatment
Component-level evaluation	Identifies failures that end-to-end accuracy can hide	Source attribution in production systems remains undermeasured

The taxonomy also highlights an important engineering reality: retrieval creates new capabilities and new ways to fail. Stale indexes, poisoned sources, query drift, reranker bias, conflicting context, and evaluator errors can all affect the path from query to answer. RAG changes where the system obtains knowledge, but it does not guarantee the quality of that knowledge.

6.2 Four limits to keep in mind in 2026

The survey successfully organized a field as it stood in 2023 and early 2024. Four limitations matter when applying its conclusions in 2026.

Claim a reader may infer	Evidence boundary	Safe wording
“This is the current RAG landscape.”	The main literature snapshot ends in early 2024	“This is a foundational map that needs to be supplemented with current research.”
“RAG reduces hallucination.”	A survey aggregates heterogeneous studies. It is not one experiment.	“RAG can improve factual grounding under particular corpora, retrievers, and evaluations.”
“Naive, Advanced, and Modular are mutually exclusive.”	The labels mix chronology, component count, workflow, and adaptation	“They are architectural patterns that can coexist in one system.”
“Production answer engines use this pipeline.”	Product indexes, rerankers, policies, and attribution layers are proprietary	“Public product behavior is consistent with some RAG mechanisms, but does not prove that the architecture is identical.”

Snapshot age. Agentic search, deep-research workflows, multimodal retrieval, long-context routing, and production web grounding developed rapidly after the paper’s last revision. Evidence about these developments should supplement the survey. The developments themselves should not be attributed to its original taxonomy.

Survey, not experiment. The paper summarizes studies that used different tasks, corpora, models, and metrics. It cannot establish a universal effect size for reducing hallucinations, and it does not test any GEO content intervention.

Mixed taxonomy axes. “Advanced” can refer to better preprocessing, stronger post-processing, or model adaptation. “Modular” can refer to replaceable components or a flexible control flow. The framework is useful for navigating the literature, but less suitable as a strict classification system.

Lab-to-production gap. Current products disclose only part of how they operate. Google now says its generative Search features use core ranking systems and may perform concurrent query fan-out (official guide). OpenAI says ChatGPT search may rewrite a request into multiple targeted queries and warns that citations can be incomplete or wrong (official help). Neither disclosure reveals the complete indexing, reranking, context-selection, or attribution process.

6.3 What remains useful and what needs newer evidence

The following update matrix preserves the survey’s value while distinguishing its conclusions from developments that require newer evidence.

2024 survey claim	2026 status	Newer evidence needed
External knowledge can update answers without retraining weights	Still relevant	Production corpus freshness and indexing behavior
Retrieval can be iterative or adaptive	Still relevant and now extended into agentic control	Agentic RAG architectures and operational failure studies
Reranking and compression matter after retrieval	Still relevant	Production traces and context-selection audits
Retrieval and generation need separate evaluation	Still relevant	Live, dynamic, source-level evaluation
RAG and fine-tuning can be combined	Still relevant	Task-specific cost, latency, and maintenance comparisons
RAG is the default answer to long inputs	Needs qualification	Long-context versus RAG and hybrid-routing experiments
Text-centric retrieval covers the main design space	Needs refresh	Multimodal retrieval benchmarks and provenance methods
Security is one future concern among many	Needs elevation	Poisoning, access control, prompt injection, and data-leakage tests

The trade-off between RAG and long context illustrates why newer evidence matters. Li et al.’s EMNLP 2024 study found that sufficiently resourced long-context models performed better on average in the settings they tested, while RAG was substantially cheaper. A hybrid router retained comparable performance at a lower cost. The finding does not make RAG obsolete. Instead, it reframes the choice between RAG and long context as a question of routing and resources.

Later evidence also provides a more precise view of evaluation and source preference. The 2024 evaluation survey separates relevance, accuracy, and faithfulness across retrieval and generation. Wan et al.’s conflicting-evidence experiment shows that, once two sources have been retrieved, topical relevance can outweigh several stylistic credibility signals. Both studies add measurements that were not part of the Gao survey.

7. Reproducibility and supporting artifacts

This narrative survey contains no original benchmark experiment to reproduce. Its reproducibility rating is therefore none, which means that experiment-level reproducibility does not apply to the study design. It does not mean that the paper has no supporting materials.

Artifact	Purpose	Reproducibility significance
arXiv versions v1–v5	Preserve the paper’s revision history	Allow readers to identify the exact March 2024 snapshot
Official RAG-Survey repository	Hosts citation information, slides, and project links	Supports provenance rather than experiment replication
OpenRAG Base	Curates papers, tasks, datasets, tools, and readings	Provides a maintained knowledge resource rather than a code-and-data package

The current arXiv record and the official repository differ in their metadata. The arXiv record lists ten authors and shows a version history from v1 on December 18, 2023, through v5 on March 27, 2024. The official repository gives a 2024 BibTeX entry that also includes Qianyu Guo. The structured metadata follows the current arXiv record. The additional name in the repository is noted separately and not added to the author list.

8. Practical implications for GEO

For GEO practitioners, the survey provides a clear model of the mechanism. A page must first be discoverable and represented in an index. A relevant unit must then enter the candidate set and survive reranking, filtering, and context limits. The generator must use that material before the product’s attribution system can credit the source.

Mechanism	Observable implication	What it does not prove
Indexing and chunking	Crawlability and coherent, answer-sized passages affect eligibility	Every engine crawls the page in real time
Candidate retrieval	Query relevance and corpus representation affect whether a page is considered	Keyword matching alone wins retrieval
Reranking and filtering	Source quality and passage specificity can affect whether content remains in context	One universal content format works for every reranker
Grounded generation	Clear, compatible evidence is easier to use	Grounding guarantees a visible citation
Attribution	Links and mentions depend on the product	A cited page influenced every nearby claim

The model shows where crawlability, specificity, chunk coherence, and source quality can affect the system, but it does not assign effect sizes to any of them. The answer loop describes the sequence in which these effects may occur, while citation-versus-mention measurement takes place after retrieval.

Evidence about specific interventions comes from studies such as Aggarwal et al. 2024. Gao et al. explain the machinery on which GEO depends, while Aggarwal et al. test content modifications within a particular generative-engine benchmark. A description of the mechanism cannot validate an intervention, and an intervention benchmark cannot describe the entire mechanism.

9. Further reading

• Generative engine explains how retrieval, grounding, generation, and attribution work together as system components.
• Answer loop traces the runtime sequence and its failure modes.
• GEO: Generative Engine Optimization tests content interventions in a generative-engine benchmark built on the kind of RAG mechanism that Gao et al. survey.
• What Evidence Do Language Models Find Convincing? measures which source a model prefers after retrieving conflicting evidence.