  Fact-checking is an essential tool to mitigate the spread of misinformation
and disinformation. We introduce the task of fact-checking in dialogue, which
is a relatively unexplored area. We construct DialFact, a testing benchmark
dataset of 22,245 annotated conversational claims, paired with pieces of
evidence from Wikipedia. There are three sub-tasks in DialFact: 1) Verifiable
claim detection task distinguishes whether a response carries verifiable
factual information; 2) Evidence retrieval task retrieves the most relevant
Wikipedia snippets as evidence; 3) Claim verification task predicts a dialogue
response to be supported, refuted, or not enough information. We found that
existing fact-checking models trained on non-dialogue data like FEVER fail to
perform well on our task, and thus, we propose a simple yet data-efficient
solution to effectively improve fact-checking performance in dialogue. We point
out unique challenges in DialFact such as handling the colloquialisms,
coreferences and retrieval ambiguities in the error analysis to shed light on
future research in this direction.
