  Pretrained language models (PLMs) have made remarkable progress in text
generation tasks via fine-tuning. While, it is challenging to fine-tune PLMs in
a data-scarce situation. Therefore, it is non-trivial to develop a general and
lightweight model that can adapt to various text generation tasks based on
PLMs. To fulfill this purpose, the recent prompt-based learning offers a
potential solution. In this paper, we improve this technique and propose a
novel prompt-based method (PTG) for text generation in a transferable setting.
First, PTG learns a set of source prompts for various source generation tasks
and then transfers these prompts as target prompts to perform target generation
tasks. To consider both task- and instance-level information, we design an
adaptive attention mechanism to derive the target prompts. For each data
instance, PTG learns a specific target prompt by attending to highly relevant
source prompts. In extensive experiments, PTG yields competitive or better
results than fine-tuning methods. We release our source prompts as an open
resource, where users can add or reuse them to improve new text generation
tasks for future research. Code and data can be available at
https://github.com/RUCAIBox/Transfer-Prompts-for-Text-Generation.
