  Dialogue contradiction is a critical issue in open-domain dialogue systems.
The contextualization nature of conversations makes dialogue contradiction
detection rather challenging. In this work, we propose a benchmark for
Contradiction Detection in Chinese Conversations, namely CDConv. It contains
12K multi-turn conversations annotated with three typical contradiction
categories: Intra-sentence Contradiction, Role Confusion, and History
Contradiction. To efficiently construct the CDConv conversations, we devise a
series of methods for automatic conversation generation, which simulate common
user behaviors that trigger chatbots to make contradictions. We conduct careful
manual quality screening of the constructed conversations and show that
state-of-the-art Chinese chatbots can be easily goaded into making
contradictions. Experiments on CDConv show that properly modeling contextual
information is critical for dialogue contradiction detection, but there are
still unresolved challenges that require future research.
