  This paper propose to combine pretrained language models with the modular
dialogue paradigm for open-domain dialogue modeling. Our method,
semantic-enhanced finetuning, instantiates conversation understanding,
planning, and response generation as a language model finetuning task. At
inference, we disentangle semantic and token variations by specifying sampling
methods and constraints for each module separately. For training and
evaluation, we present X-Weibo, a Chinese multi-turn open-domain dialogue
dataset with automatic annotation for emotions, DAs, and topical words.
Experiments show that semantic-enhanced finetuning outperforms strong baselines
on non-semantic and semantic metrics, improves the human-evaluated relevance,
coherence, and informativeness, and exhibits considerable controllability over
semantic variables.
