  Large language models (LLMs) with retrieval augmented-generation (RAG) have
been the optimal choice for scalable generative AI solutions in the recent
past. However, the choice of use-cases that incorporate RAG with LLMs have been
either generic or extremely domain specific, thereby questioning the
scalability and generalizability of RAG-LLM approaches. In this work, we
propose a unique LLM-based system where multiple LLMs can be invoked to enable
data authentication, user query routing, data retrieval and custom prompting
for question answering capabilities from data tables that are highly varying
and large in size. Our system is tuned to extract information from
Enterprise-level data products and furnish real time responses under 10
seconds. One prompt manages user-to-data authentication followed by three
prompts to route, fetch data and generate a customizable prompt natural
language responses. Additionally, we propose a five metric scoring module that
detects and reports hallucinations in the LLM responses. Our proposed system
and scoring metrics achieve >90% confidence scores across hundreds of user
queries in the sustainability, financial health and social media domains.
Extensions to the proposed extreme RAG architectures can enable heterogeneous
source querying using LLMs.
