  Recently, various neural models for multi-party conversation (MPC) have
achieved impressive improvements on a variety of tasks such as addressee
recognition, speaker identification and response prediction. However, these
existing methods on MPC usually represent interlocutors and utterances
individually and ignore the inherent complicated structure in MPC which may
provide crucial interlocutor and utterance semantics and would enhance the
conversation understanding process. To this end, we present MPC-BERT, a
pre-trained model for MPC understanding that considers learning who says what
to whom in a unified model with several elaborated self-supervised tasks.
Particularly, these tasks can be generally categorized into (1) interlocutor
structure modeling including reply-to utterance recognition, identical speaker
searching and pointer consistency distinction, and (2) utterance semantics
modeling including masked shared utterance restoration and shared node
detection. We evaluate MPC-BERT on three downstream tasks including addressee
recognition, speaker identification and response selection. Experimental
results show that MPC-BERT outperforms previous methods by large margins and
achieves new state-of-the-art performance on all three downstream tasks at two
benchmarks.
