  Large Language Models (LLMs) have shown impressive capabilities but also a
concerning tendency to hallucinate. This paper presents RefChecker, a framework
that introduces claim-triplets to represent claims in LLM responses, aiming to
detect fine-grained hallucinations. In RefChecker, an extractor generates
claim-triplets from a response, which are then evaluated by a checker against a
reference. We delineate three task settings: Zero, Noisy and Accurate Context,
to reflect various real-world use cases. We curated a benchmark spanning
various NLP tasks and annotated 11k claim-triplets from 2.1k responses by seven
LLMs. RefChecker supports both proprietary and open-source models as the
extractor and checker. Experiments demonstrate that claim-triplets enable
superior hallucination detection, compared to other granularities such as
response, sentence and sub-sentence level claims. RefChecker outperforms prior
methods by 6.8 to 26.1 points on our benchmark and the checking results of
RefChecker are strongly aligned with human judgments. This work is open sourced
at https://github.com/amazon-science/RefChecker
