  The goal of this work is to build conversational Question Answering (QA)
interfaces for the large body of domain-specific information available in FAQ
sites. We present DoQA, a dataset with 2,437 dialogues and 10,917 QA pairs. The
dialogues are collected from three Stack Exchange sites using the Wizard of Oz
method with crowdsourcing. Compared to previous work, DoQA comprises
well-defined information needs, leading to more coherent and natural
conversations with less factoid questions and is multi-domain. In addition, we
introduce a more realistic information retrieval(IR) scenario where the system
needs to find the answer in any of the FAQ documents. The results of an
existing, strong, system show that, thanks to transfer learning from a
Wikipedia QA dataset and fine tuning on a single FAQ domain, it is possible to
build high quality conversational QA systems for FAQs without in-domain
training data. The good results carry over into the more challenging IR
scenario. In both cases, there is still ample room for improvement, as
indicated by the higher human upperbound.
