  Current task-oriented dialog (TOD) systems mostly manage structured knowledge
(e.g. databases and tables) to guide the goal-oriented conversations. However,
they fall short of handling dialogs which also involve unstructured knowledge
(e.g. reviews and documents). In this paper, we formulate a task of modeling
TOD grounded on a fusion of structured and unstructured knowledge. To address
this task, we propose a TOD system with semi-structured knowledge management,
SeKnow, which extends the belief state to manage knowledge with both structured
and unstructured contents. Furthermore, we introduce two implementations of
SeKnow based on a non-pretrained sequence-to-sequence model and a pretrained
language model, respectively. Both implementations use the end-to-end manner to
jointly optimize dialog modeling grounded on structured and unstructured
knowledge. We conduct experiments on a modified version of MultiWOZ 2.1
dataset, Mod-MultiWOZ 2.1, where dialogs are processed to involve
semi-structured knowledge. Experimental results show that SeKnow has strong
performances in both end-to-end dialog and intermediate knowledge management,
compared to existing TOD systems and their extensions with pipeline knowledge
management schemes.
