  To hold a true conversation, an intelligent agent should be able to
occasionally take initiative and recommend the next natural conversation topic.
This is a challenging task. A topic suggested by the agent should be relevant
to the person, appropriate for the conversation context, and the agent should
have something interesting to say about it. Thus, a scripted, or
one-size-fits-all, popularity-based topic suggestion is doomed to fail.
Instead, we explore different methods for a personalized, contextual topic
suggestion for open-domain conversations. We formalize the Conversational Topic
Suggestion problem (CTS) to more clearly identify the assumptions and
requirements. We also explore three possible approaches to solve this problem:
(1) model-based sequential topic suggestion to capture the conversation context
(CTS-Seq), (2) Collaborative Filtering-based suggestion to capture previous
successful conversations from similar users (CTS-CF), and (3) a hybrid approach
combining both conversation context and collaborative filtering. To evaluate
the effectiveness of these methods, we use real conversations collected as part
of the Amazon Alexa Prize 2018 Conversational AI challenge. The results are
promising: the CTS-Seq model suggests topics with 23% higher accuracy than the
baseline, and incorporating collaborative filtering signals into a hybrid
CTS-Seq-CF model further improves recommendation accuracy by 12%. Together, our
proposed models, experiments, and analysis significantly advance the study of
open-domain conversational agents, and suggest promising directions for future
improvements.
