  A pressing challenge in current dialogue systems is to successfully converse
with users on topics with information distributed across different modalities.
Previous work in multiturn dialogue systems has primarily focused on either
text or table information. In more realistic scenarios, having a joint
understanding of both is critical as knowledge is typically distributed over
both unstructured and structured forms. We present a new dialogue dataset,
HybriDialogue, which consists of crowdsourced natural conversations grounded on
both Wikipedia text and tables. The conversations are created through the
decomposition of complex multihop questions into simple, realistic multiturn
dialogue interactions. We propose retrieval, system state tracking, and
dialogue response generation tasks for our dataset and conduct baseline
experiments for each. Our results show that there is still ample opportunity
for improvement, demonstrating the importance of building stronger dialogue
systems that can reason over the complex setting of information-seeking
dialogue grounded on tables and text.
