  Retriever Augmented Generation (RAG) systems have become pivotal in enhancing
the capabilities of language models by incorporating external knowledge
retrieval mechanisms. However, a significant challenge in deploying these
systems in industry applications is the detection and mitigation of
hallucinations: instances where the model generates information that is not
grounded in the retrieved context. Addressing this issue is crucial for
ensuring the reliability and accuracy of responses generated by large language
models (LLMs) in diverse industry settings. Current hallucination detection
techniques fail to deliver accuracy, low latency, and low cost simultaneously.
We introduce Luna: a DeBERTA-large (440M) encoder, finetuned for hallucination
detection in RAG settings. We demonstrate that Luna outperforms GPT-3.5 and
commercial evaluation frameworks on the hallucination detection task, with 97%
and 91% reduction in cost and latency, respectively. Luna is lightweight and
generalizes across multiple industry verticals and out-of-domain data, making
it an ideal candidate for industry LLM applications.
