  Hallucination, a phenomenon where large language models (LLMs) produce output
that is factually incorrect or unrelated to the input, is a major challenge for
LLM applications that require accuracy and dependability. In this paper, we
introduce a reliable and high-speed production system aimed at detecting and
rectifying the hallucination issue within LLMs. Our system encompasses named
entity recognition (NER), natural language inference (NLI), span-based
detection (SBD), and an intricate decision tree-based process to reliably
detect a wide range of hallucinations in LLM responses. Furthermore, our team
has crafted a rewriting mechanism that maintains an optimal mix of precision,
response time, and cost-effectiveness. We detail the core elements of our
framework and underscore the paramount challenges tied to response time,
availability, and performance metrics, which are crucial for real-world
deployment of these technologies. Our extensive evaluation, utilizing offline
data and live production traffic, confirms the efficacy of our proposed
framework and service.
