Language model search tools deliver smaller gains when combined
Three retrieval techniques predicted a 47% combined improvement but delivered only 17% when used together across eight benchmarks.
Edited by Boris Djordjevic · Paperfoot
The authors introduced LASAGNA, an open-source evaluation framework for retrieval-augmented generation, in a preprint evaluating 450 pipeline configurations across eight benchmarks and document collections ranging from 5,000 to 200,000 documents. Retrieval-augmented generation grounds language models in external knowledge by retrieving supporting text. Modern systems combine steps such as search, reranking, and compression, but engineers usually test these parts in isolation.
The authors evaluated three pre-declared hypotheses. Individually strong techniques did not necessarily combine well. Three retrieval techniques predicted a 47% combined improvement, but delivered only 17% when used together. Gains also stacked differently for retrieval quality and answer faithfulness, and optimizing one can hurt the other. Finally, the best combination changed as collections grew. A design that won on a small corpus may not remain best at large scale.
Why it matters
The findings show that evaluating search components in isolation does not reliably predict how they perform inside a complete system. Across 450 configurations on eight benchmarks, the work provides an open-source framework for developers to test how different pipeline choices interact before deploying them.
Caveats
The abstract does not report which language models were evaluated, their sizes, or whether results varied across runs. The paper is a preprint that has not yet been peer-reviewed.
In this paper
controlled RAG evaluation
- Methods
- LASAGNA (this paper)balanced (this paper)standard (this paper)speed (this paper)max-reliable (this paper)default_standard (baseline)BM25-only retriever (baseline)MMR (evaluated)Matryoshka truncation of BGE-M3 (evaluated)HyDE (evaluated)2019 MS-MARCO MiniLM-L6 cross-encoder (evaluated)GPT-5.1
- Datasets
- NFCorpus (evaluated)FiQA (evaluated)SciFact (evaluated)ArguAna (evaluated)HotpotQA (evaluated)SQuAD v2 (evaluated)MuSiQue (evaluated)
- Links
- Code
Results
- default_standard, NDCG@10: 0.412
The three largest separately measured retrieval gains, from cross-encoder reranking, disabling MMR, and document-awareness prefixes, predict a +46.6% lift over the default_standard baseline (NDCG@10 =0.412; Table 32).
H1: the benefit of a RAG technique depends on the complete RAG pipeline and the target metric.
- BM25-only retriever, NDCG@10, On this same split: 0.403
On this same split a BM25-only retriever scores 0.403 and a recommended off-the-shelf dense encoder 0.436 (Table 60), so this is a realistic starting point, not a weak one.
H1: the benefit of a RAG technique depends on the complete RAG pipeline and the target metric.
- balanced, retrieval gain: +17.2%
The balanced pipeline, which combines these techniques, realizes only +17.2%, a 63% shortfall relative to the three-technique prediction.
H1: the benefit of a RAG technique depends on the complete RAG pipeline and the target metric.
- balanced, NDCG@10, At 200K: 13.7% relative
At 200K, balanced exceeds standard by 13.7% relative ( p=0.003; full per-size table in Section E.1).
H3: composition rankings shift between 5K and 200K documents.
- standard, joint hypervolume, At the fixed 30 s response-time reference: 0.523
At the fixed 30 s response-time reference, standard has the highest joint hypervolume ( 0.523 ).
H2: complete RAG profiles expose a Pareto tradeoff.
- balanced, NDCG@10, Within the agentic pair: 4.5%; balanced, response time, Within the agentic pair: 13%
Within the agentic pair, balanced improves NDCG@10 by 4.5% and response time by 13%, with only a 0.005 faithfulness difference.
H2: complete RAG profiles expose a Pareto tradeoff.
- MMR, NDCG@10, on average: 3.6% relative
MMR reduces NDCG@10 by 3.6% relative on average, opposite its usual benefit on broad, heterogeneous corpora (Table 44).
H1: retrieval quality is most strongly associated with embedding and enrichment.
- Matryoshka truncation of BGE-M3, NDCG@10, to 768 dimensions: 2.3%
Matryoshka truncation of BGE-M3 to 768 dimensions reduces vector storage and search cost by 25% while improving NDCG@10 by 2.3% (Table 52).
Supporting evidence: what helps and what does not.
- HyDE, NDCG@10: 3.1%
HyDE raises NDCG@10 by 3.1% but doubles first-hop latency, reducing the latency-adjusted gain to 0.9%, so its value depends on the latency budget (Section I.2).
Supporting evidence: what helps and what does not.
What it shows
- Individually strong techniques from different phases of a RAG pipeline do not necessarily compose, delivering far less combined improvement than predicted.
First, individually strong techniques from different phases of a RAG pipeline do not necessarily compose: three retrieval techniques predict a +47% combined improvement but deliver only +17% when used together.
Abstract
- Component gains stack differently for retrieval and faithfulness, and optimizing one metric can hurt the other.
Second, component gains stack differently for retrieval and faithfulness, and optimizing one can hurt the other.
Abstract
- The best RAG composition changes as corpus scale increases, so configurations optimal on small corpora may not remain best at larger scale.
Third, the best composition changes as the corpus grows: the combination that wins on a small corpus may not remain best at large scale.
Abstract
- Under a fixed thirty-second response-time reference, the standard profile achieves the highest joint quality-latency hypervolume.
At the fixed 30 s response-time reference, standard has the highest joint hypervolume ( 0.523 ).
H2: complete RAG profiles expose a Pareto tradeoff.
Quoted word for word from the full text on arXiv.
The paper
Retrieval Augmented Generation Compositionality
arXiv, Oct 21, 2026, Open access, CC BY, Preprint, not peer-reviewed
doi.org/10.48550/arxiv.2610.29722
arXiv:2610.29722v1 in cs.CL, also cs.AI
Comments: 103 pages (9 main), 15 figures, 68 tables. Project page: https://dolev31.github.io/lasagna/ Code: https://github.com/dolev31/lasagna
Cite as BibTeX
@misc{levy2026retrieval,
title = {Retrieval Augmented Generation Compositionality},
author = {Levy, Ido and Shlomov, Segev and Choshen, Leshem},
year = {2026},
eprint = {2610.29722},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
doi = {10.48550/arxiv.2610.29722},
url = {https://arxiv.org/abs/2610.29722}
}