MearcAll papers

arXiv 2005.11401 · cs.CL cs.LG

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, and 6 more

Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.

No reproduction recordedclm_167777c7
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks.

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_ed50a2f4
We fine-tune and evaluate our models on a wide range of knowledgeintensive NLP tasks and set the state of the art on three open domain QA tasks, outperforming parametric seq2seq … (excerpt)

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_bc1aaab8
For language generation tasks, we find that RAG models generate more specific, diverse and factual language than a state-of-the-art parametric-only seq2seq baseline.

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_6e3bce29
Our RAG models achieve state-of-the-art results on open Natural Questions [29], WebQuestions [3] and CuratedTrec [2] and strongly outperform recent approaches that use specialised pre-training objectives on TriviaQA [24].

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_ece08d85
For FEVER [56] fact verification, we achieve results within 4.3% of state-of-the-art pipeline models which use strong retrieval supervision.

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_dfbaeed6
Finally, we demonstrate that the non-parametric memory can be replaced to update the models’ knowledge as the world changes.1

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_74f2592c
We propose two models that marginalize over the latent documents in different ways to produce a distribution over generated text.

Methods · page 3 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_bbbe7cd8
Rather than use questions from standard open-domain QA tasks, which typically consist of short, simple questions, we propose the more demanding task of generating Jeopardy questions.

Jeopardy Question Generation · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_56ba7c43
On all four open-domain QA tasks, RAG sets a new state of the art (only on the T5-comparable split for TQA).

Open-domain Question Answering · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_57e9dbb6
Furthermore, RAG can generate correct answers even when the correct answer is not in any retrieved document, achieving 11.8% accuracy in such cases for NQ, where an extractive model would … (excerpt)

Open-domain Question Answering · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_a024f45a
As shown in Table 2, RAG-Sequence outperforms BART on Open MS-MARCO NLG by 2.6 Bleu points and 2.6 Rouge-L points.

Abstractive Question Answering · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_cd79682d
Table 2 shows that RAG-Token performs better than RAG-Sequence on Jeopardy question generation, with both models outperforming BART on Q-BLEU-1.

Jeopardy Question Generation · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_5f94924e
RAG achieves an accuracy within 2.7% of this model, despite being supplied with only the claim and retrieving its own evidence.

Jeopardy Question Generation · page 7 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_67d24663
As shown in Table 6, learned retrieval improves results for all tasks.

Additional Results · page 7 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_276c033d
Figure 3 (left) shows that retrieving more documents at test time monotonically improves Open-domain QA results for RAG-Sequence, but performance peaks for RAG-Token at 10 retrieved documents.

Additional Results · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_376c5e5e
Figure 3 (right) shows that retrieving more documents leads to higher Rouge-L for RAG-Token at the expense of Bleu-1, but the effect is less pronounced for RAG-Sequence.

Additional Results · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_c01ca085
We showed that our RAG models obtain state of the art results on open-domain QA.

Discussion · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_cc2b7149
After submission, We have ported our code to HuggingFace Transformers [66]3, which achieves equivalent performance to the previous version but is a cleaner and easier to use implementation.

Training setup Details · page 17 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_ec63024d
The T5 model with the closest number of parameters to our models is T5-large (770M parameters), which achieves a score of 28.9 EM on Natural Questions [52], substantially below the … (excerpt)

Parameters · page 19 · Automated extraction, not reviewed · 0 reviewed reproductions

Reproduced one of these claims?

Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (2005.11401) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.