  Neural models trained with large amount of parallel data have achieved
impressive performance in abstractive summarization tasks. However, large-scale
parallel corpora are expensive and challenging to construct. In this work, we
introduce a low-cost and effective strategy, ExtraPhrase, to augment training
data for abstractive summarization tasks. ExtraPhrase constructs pseudo
training data in two steps: extractive summarization and paraphrasing. We
extract major parts of an input text in the extractive summarization step, and
obtain its diverse expressions with the paraphrasing step. Through experiments,
we show that ExtraPhrase improves the performance of abstractive summarization
tasks by more than 0.50 points in ROUGE scores compared to the setting without
data augmentation. ExtraPhrase also outperforms existing methods such as
back-translation and self-training. We also show that ExtraPhrase is
significantly effective when the amount of genuine training data is remarkably
small, i.e., a low-resource setting. Moreover, ExtraPhrase is more
cost-efficient than the existing approaches.
