  Query-focused summarization has been considered as an important extension for
text summarization. It aims to generate a concise highlight for a given query.
Different from text summarization, query-focused summarization has long been
plagued by the problem of lacking high-quality large-scale datasets. In this
paper, we investigate the idea that whether we can integrate and transfer the
knowledge of text summarization and question answering to assist the few-shot
learning in query-focused summarization. Here, we propose prefix-merging, a
prefix-based pretraining strategy for few-shot learning in query-focused
summarization. Drawn inspiration from prefix-tuning, we are allowed to
integrate the task knowledge from text summarization and question answering
into a properly designed prefix and apply the merged prefix to query-focused
summarization. With only a small amount of trainable parameters, prefix-merging
outperforms fine-tuning on query-focused summarization. We further discuss the
influence of different prefix designs and propose a visualized explanation for
how prefix-merging works.
