  We introduce a novel graph-based Retrieval-Augmented Generation (RAG)
framework specifically designed for the medical domain, called
\textbf{MedGraphRAG}, aimed at enhancing Large Language Model (LLM)
capabilities and generating evidence-based results, thereby improving safety
and reliability when handling private medical data. Our comprehensive pipeline
begins with a hybrid static-semantic approach to document chunking,
significantly improving context capture over traditional methods. Extracted
entities are used to create a three-tier hierarchical graph structure, linking
entities to foundational medical knowledge sourced from medical papers and
dictionaries. These entities are then interconnected to form meta-graphs, which
are merged based on semantic similarities to develop a comprehensive global
graph. This structure supports precise information retrieval and response
generation. The retrieval process employs a U-retrieve method to balance global
awareness and indexing efficiency of the LLM. Our approach is validated through
a comprehensive ablation study comparing various methods for document chunking,
graph construction, and information retrieval. The results not only demonstrate
that our hierarchical graph construction method consistently outperforms
state-of-the-art models on multiple medical Q\&A benchmarks, but also confirms
that the responses generated include source documentation, significantly
enhancing the reliability of medical LLMs in practical applications. Code will
be at: https://github.com/MedicineToken/Medical-Graph-RAG/tree/main
