  The rise of increasingly more powerful chatbots offers a new way to collect
information through conversational surveys, where a chatbot asks open-ended
questions, interprets a user's free-text responses, and probes answers whenever
needed. To investigate the effectiveness and limitations of such a chatbot in
conducting surveys, we conducted a field study involving about 600
participants. In this study with mostly open-ended questions, half of the
participants took a typical online survey on Qualtrics and the other half
interacted with an AI-powered chatbot to complete a conversational survey. Our
detailed analysis of over 5200 free-text responses revealed that the chatbot
drove a significantly higher level of participant engagement and elicited
significantly better quality responses measured by Gricean Maxims in terms of
their informativeness, relevance, specificity, and clarity. Based on our
results, we discuss design implications for creating AI-powered chatbots to
conduct effective surveys and beyond.
