  In spoken question answering, QA systems are designed to answer questions
from contiguous text spans within the related speech transcripts. However, the
most natural way that human seek or test their knowledge is via human
conversations. Therefore, we propose a new Spoken Conversational Question
Answering task (SCQA), aiming at enabling QA systems to model complex dialogues
flow given the speech utterances and text corpora. In this task, our main
objective is to build a QA system to deal with conversational questions both in
spoken and text forms, and to explore the plausibility of providing more cues
in spoken documents with systems in information gathering. To this end, instead
of adopting automatically generated speech transcripts with highly noisy data,
we propose a novel unified data distillation approach, DDNet, which directly
fuse audio-text features to reduce the misalignment between automatic speech
recognition hypotheses and the reference transcriptions. In addition, to
evaluate the capacity of QA systems in a dialogue-style interaction, we
assemble a Spoken Conversational Question Answering (Spoken-CoQA) dataset with
more than 120k question-answer pairs. Experiments demonstrate that our proposed
method achieves superior performance in spoken conversational question
answering.
