  The goal of information-seeking dialogue is to respond to seeker queries with
natural language utterances that are grounded on knowledge sources. However,
dialogue systems often produce unsupported utterances, a phenomenon known as
hallucination. To mitigate this behavior, we adopt a data-centric solution and
create FaithDial, a new benchmark for hallucination-free dialogues, by editing
hallucinated responses in the Wizard of Wikipedia (WoW) benchmark. We observe
that FaithDial is more faithful than WoW while also maintaining engaging
conversations. We show that FaithDial can serve as training signal for: i) a
hallucination critic, which discriminates whether an utterance is faithful or
not, and boosts the performance by 12.8 F1 score on the BEGIN benchmark
compared to existing datasets for dialogue coherence; ii) high-quality dialogue
generation. We benchmark a series of state-of-the-art models and propose an
auxiliary contrastive objective that achieves the highest level of faithfulness
and abstractiveness based on several automated metrics. Further, we find that
the benefits of FaithDial generalize to zero-shot transfer on other datasets,
such as CMU-Dog and TopicalChat. Finally, human evaluation reveals that
responses generated by models trained on FaithDial are perceived as more
interpretable, cooperative, and engaging.
