  Task-oriented dialog (TOD) systems typically manage structured knowledge
(e.g. ontologies and databases) to guide the goal-oriented conversations.
However, they fall short of handling dialog turns grounded on unstructured
knowledge (e.g. reviews and documents). In this paper, we formulate a task of
modeling TOD grounded on both structured and unstructured knowledge. To address
this task, we propose a TOD system with hybrid knowledge management, HyKnow. It
extends the belief state to manage both structured and unstructured knowledge,
and is the first end-to-end model that jointly optimizes dialog modeling
grounded on these two kinds of knowledge. We conduct experiments on the
modified version of MultiWOZ 2.1 dataset, where dialogs are grounded on hybrid
knowledge. Experimental results show that HyKnow has strong end-to-end
performance compared to existing TOD systems. It also outperforms the pipeline
knowledge management schemes, with higher unstructured knowledge retrieval
accuracy.
