  We study knowledge-grounded dialogue generation with pre-trained language
models. Instead of pursuing new state-of-the-art on benchmarks, we try to
understand if the knowledge stored in parameters of the pre-trained models is
already enough to ground open domain dialogues, and thus allows us to get rid
of the dependency on external knowledge sources in generation. Through
extensive experiments on benchmarks, we find that by fine-tuning with a few
dialogues containing knowledge, the pre-trained language models can outperform
the state-of-the-art model that requires external knowledge in automatic
evaluation and human judgment, suggesting a positive answer to the question we
raised.
