  Existing summarization systems mostly generate summaries purely relying on
the content of the source document. However, even for humans, we usually need
some references or exemplars to help us fully understand the source document
and write summaries in a particular format. But how to find the high-quality
exemplars and incorporate them into summarization systems is still challenging
and worth exploring. In this paper, we propose RetrievalSum, a novel retrieval
enhanced abstractive summarization framework consisting of a dense Retriever
and a Summarizer. At first, several closely related exemplars are retrieved as
supplementary input to help the generation model understand the text more
comprehensively. Furthermore, retrieved exemplars can also play a role in
guiding the model to capture the writing style of a specific corpus. We
validate our method on a wide range of summarization datasets across multiple
domains and two backbone models: BERT and BART. Results show that our framework
obtains significant improvement by 1.38~4.66 in ROUGE-1 score when compared
with the powerful pre-trained models, and achieve new state-of-the-art on
BillSum. Human evaluation demonstrates that our retrieval enhanced model can
better capture the domain-specific writing style.
