  We present MILABOT: a deep reinforcement learning chatbot developed by the
Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize
competition. MILABOT is capable of conversing with humans on popular small talk
topics through both speech and text. The system consists of an ensemble of
natural language generation and retrieval models, including neural network and
template-based models. By applying reinforcement learning to crowdsourced data
and real-world user interactions, the system has been trained to select an
appropriate response from the models in its ensemble. The system has been
evaluated through A/B testing with real-world users, where it performed
significantly better than other systems. The results highlight the potential of
coupling ensemble systems with deep reinforcement learning as a fruitful path
for developing real-world, open-domain conversational agents.
