  Large language models (LLMs) have advanced the field of artificial
intelligence (AI) in medicine. However LLMs often generate outdated or
inaccurate information based on static training datasets. Retrieval augmented
generation (RAG) mitigates this by integrating outside data sources. While
previous RAG systems used pre-assembled, fixed databases with limited
flexibility, we have developed Radiology RAG (RadioRAG) as an end-to-end
framework that retrieves data from authoritative radiologic online sources in
real-time. RadioRAG is evaluated using a dedicated radiologic
question-and-answer dataset (RadioQA). We evaluate the diagnostic accuracy of
various LLMs when answering radiology-specific questions with and without
access to additional online information via RAG. Using 80 questions from RSNA
Case Collection across radiologic subspecialties and 24 additional
expert-curated questions, for which the correct gold-standard answers were
available, LLMs (GPT-3.5-turbo, GPT-4, Mistral-7B, Mixtral-8x7B, and Llama3 [8B
and 70B]) were prompted with and without RadioRAG. RadioRAG retrieved
context-specific information from www.radiopaedia.org in real-time and
incorporated them into its reply. RadioRAG consistently improved diagnostic
accuracy across all LLMs, with relative improvements ranging from 2% to 54%. It
matched or exceeded question answering without RAG across radiologic
subspecialties, particularly in breast imaging and emergency radiology.
However, degree of improvement varied among models; GPT-3.5-turbo and
Mixtral-8x7B-instruct-v0.1 saw notable gains, while Mistral-7B-instruct-v0.2
showed no improvement, highlighting variability in its effectiveness. LLMs
benefit when provided access to domain-specific data beyond their training
data. For radiology, RadioRAG establishes a robust framework that substantially
improves diagnostic accuracy and factuality in radiological question answering.
