  Training machines to understand natural language and interact with humans is
one of the major goals of artificial intelligence. Recent years have witnessed
an evolution from matching networks to pre-trained language models (PrLMs). In
contrast to the plain-text modeling as the focus of the PrLMs, dialogue texts
involve multiple speakers and reflect special characteristics such as topic
transitions and structure dependencies between distant utterances. However, the
related PrLM models commonly represent dialogues sequentially by processing the
pairwise dialogue history as a whole. Thus the hierarchical information on
either utterance interrelation or speaker roles coupled in such representations
is not well addressed. In this work, we propose compositional learning for
holistic interaction across the utterances beyond the sequential
contextualization from PrLMs, in order to capture the utterance-aware and
speaker-aware representations entailed in a dialogue history. We decouple the
contextualized word representations by masking mechanisms in Transformer-based
PrLM, making each word only focus on the words in current utterance, other
utterances, and two speaker roles (i.e., utterances of sender and utterances of
the receiver), respectively. In addition, we employ domain-adaptive training
strategies to help the model adapt to the dialogue domains. Experimental
results show that our method substantially boosts the strong PrLM baselines in
four public benchmark datasets, achieving new state-of-the-art performance over
previous methods.
