  Open domain conversational agents can answer a broad range of targeted
queries. However, the sequential nature of interaction with these systems makes
knowledge exploration a lengthy task which burdens the user with asking a chain
of well phrased questions. In this paper, we present a retrieval based system
and associated dataset for predicting the next questions that the user might
have. Such a system can proactively assist users in knowledge exploration
leading to a more engaging dialog. The retrieval system is trained on a dataset
which contains ~14K multi-turn information-seeking conversations with a valid
follow-up question and a set of invalid candidates. The invalid candidates are
generated to simulate various syntactic and semantic confounders such as
paraphrases, partial entity match, irrelevant entity, and ASR errors. We use
confounder specific techniques to simulate these negative examples on the
OR-QuAC dataset and develop a dataset called the Follow-up Query Bank
(FQ-Bank). Then, we train ranking models on FQ-Bank and present results
comparing supervised and unsupervised approaches. The results suggest that we
can retrieve the valid follow-ups by ranking them in higher positions compared
to confounders, but further knowledge grounding can improve ranking
performance.
