  State-of-the-art conversational agents have advanced significantly in
conjunction with the use of large transformer-based language models. However,
even with these advancements, conversational agents still lack the ability to
produce responses that are informative and coherent with the local context. In
this work, we propose a dialog framework that incorporates both local knowledge
as well as users' past dialogues to generate high quality conversations. We
introduce an approach to build a dataset based on Reddit conversations, where
outbound URL links are widely available in the conversations and the
hyperlinked documents can be naturally included as local external knowledge.
Using our framework and dataset, we demonstrate that incorporating local
knowledge can largely improve informativeness, coherency and realisticness
measures using human evaluations. In particular, our approach consistently
outperforms the state-of-the-art conversational model on the Reddit dataset
across all three measures. We also find that scaling the size of our models
from 117M to 8.3B parameters yields consistent improvement of validation
perplexity as well as human evaluated metrics. Our model with 8.3B parameters
can generate human-like responses as rated by various human evaluations in a
single-turn dialog setting.
