  In this paper, we study the problem of addressee and response selection in
multi-party conversations. Understanding multi-party conversations is
challenging because of complex speaker interactions: multiple speakers exchange
messages with each other, playing different roles (sender, addressee,
observer), and these roles vary across turns. To tackle this challenge, we
propose the Speaker Interaction Recurrent Neural Network (SI-RNN). Whereas the
previous state-of-the-art system updated speaker embeddings only for the
sender, SI-RNN uses a novel dialog encoder to update speaker embeddings in a
role-sensitive way. Additionally, unlike the previous work that selected the
addressee and response separately, SI-RNN selects them jointly by viewing the
task as a sequence prediction problem. Experimental results show that SI-RNN
significantly improves the accuracy of addressee and response selection,
particularly in complex conversations with many speakers and responses to
distant messages many turns in the past.
