  Existing studies in dialogue system research mostly treat task-oriented
dialogue and chit-chat as separate domains. Towards building a human-like
assistant that can converse naturally and seamlessly with users, it is
important to build a dialogue system that conducts both types of conversations
effectively. In this work, we investigate how task-oriented dialogue and
knowledge-grounded chit-chat can be effectively integrated into a single model.
To this end, we create a new dataset, KETOD (Knowledge-Enriched Task-Oriented
Dialogue), where we naturally enrich task-oriented dialogues with chit-chat
based on relevant entity knowledge. We also propose two new models,
SimpleToDPlus and Combiner, for the proposed task. Experimental results on both
automatic and human evaluations show that the proposed methods can
significantly improve the performance in knowledge-enriched response generation
while maintaining a competitive task-oriented dialog performance. We believe
our new dataset will be a valuable resource for future studies. Our dataset and
code are publicly available at \url{https://github.com/facebookresearch/ketod}.
