  Open-domain conversational search (ODCS) aims to provide valuable, up-to-date
information, while maintaining natural conversations to help users refine and
ultimately answer information needs. However, creating an effective and robust
ODCS agent is challenging. In this paper, we present a fully functional ODCS
system, Ericson, which includes state-of-the-art question answering and
information retrieval components, as well as intent inference and dialogue
management models for proactive question refinement and recommendations. Our
system was stress-tested in the Amazon Alexa Prize, by engaging in live
conversations with thousands of Alexa users, thus providing empirical basis for
the analysis of the ODCS system in real settings. Our interaction data analysis
revealed that accurate intent classification, encouraging user engagement, and
careful proactive recommendations contribute most to the users satisfaction.
Our study further identifies limitations of the existing search techniques, and
can serve as a building block for the next generation of ODCS agents.
