  Learning sentence embeddings from dialogues has drawn increasing attention
due to its low annotation cost and high domain adaptability. Conventional
approaches employ the siamese-network for this task, which obtains the sentence
embeddings through modeling the context-response semantic relevance by applying
a feed-forward network on top of the sentence encoders. However, as the
semantic textual similarity is commonly measured through the element-wise
distance metrics (e.g. cosine and L2 distance), such architecture yields a
large gap between training and evaluating. In this paper, we propose
DialogueCSE, a dialogue-based contrastive learning approach to tackle this
issue. DialogueCSE first introduces a novel matching-guided embedding (MGE)
mechanism, which generates a context-aware embedding for each candidate
response embedding (i.e. the context-free embedding) according to the guidance
of the multi-turn context-response matching matrices. Then it pairs each
context-aware embedding with its corresponding context-free embedding and
finally minimizes the contrastive loss across all pairs. We evaluate our model
on three multi-turn dialogue datasets: the Microsoft Dialogue Corpus, the Jing
Dong Dialogue Corpus, and the E-commerce Dialogue Corpus. Evaluation results
show that our approach significantly outperforms the baselines across all three
datasets in terms of MAP and Spearman's correlation measures, demonstrating its
effectiveness. Further quantitative experiments show that our approach achieves
better performance when leveraging more dialogue context and remains robust
when less training data is provided.
