  We propose a new paradigm to help Large Language Models (LLMs) generate more
accurate factual knowledge without retrieving from an external corpus, called
RECITation-augmented gEneration (RECITE). Different from retrieval-augmented
language models that retrieve relevant documents before generating the outputs,
given an input, RECITE first recites one or several relevant passages from
LLMs' own memory via sampling, and then produces the final answers. We show
that RECITE is a powerful paradigm for knowledge-intensive NLP tasks.
Specifically, we show that by utilizing recitation as the intermediate step, a
recite-and-answer scheme can achieve new state-of-the-art performance in
various closed-book question answering (CBQA) tasks. In experiments, we verify
the effectiveness of \method~on four pre-trained models (PaLM, UL2, OPT, and
Codex) and three CBQA tasks (Natural Questions, TriviaQA, and HotpotQA). Our
code is available at "https://github.com/Edward-Sun/RECITE".
