  Pre-trained language models (PLM) have marked a huge leap in neural dialogue
modeling. While PLMs are pre-trained on large-scale text corpora, they are
usually fine-tuned on scarce dialogue data with specific domain knowledge and
dialogue styles. However, tailoring the language models while fully utilizing
prior knowledge in large pre-trained models remains a challenge. In this paper,
we present a novel approach for pre-trained dialogue modeling that casts the
dialogue generation problem as a prompt-learning task. Instead of fine-tuning
on limited dialogue data, our approach, DialogPrompt, learns continuous prompt
embeddings optimized for dialogue contexts, which appropriately elicit
knowledge from the large pre-trained model. To encourage the model to better
utilize the prompt embeddings, the prompt encoders are designed to be
dynamically generated based on the dialogue context. Experiments on popular
conversation datasets show that our approach significantly outperforms the
fine-tuning baseline and the generic prompt-learning methods. Furthermore,
human evaluations strongly support the superiority of DialogPrompt in regard to
response generation quality.
