  Large Language Models (LLMs) are notorious for blending fact with fiction and
generating non-factual content, known as hallucinations. To address this
challenge, we propose an interactive system that helps users gain insight into
the reliability of the generated text. Our approach is based on the idea that
the self-consistency of multiple samples generated by the same LLM relates to
its confidence in individual claims in the generated texts. Using this idea, we
design RELIC, an interactive system that enables users to investigate and
verify semantic-level variations in multiple long-form responses. This allows
users to recognize potentially inaccurate information in the generated text and
make necessary corrections. From a user study with ten participants, we
demonstrate that our approach helps users better verify the reliability of the
generated text. We further summarize the design implications and lessons
learned from this research for future studies of reliable human-LLM
interactions.
