  Large Language Models (LLMs) have shown great potential in the biomedical
domain with the advancement of retrieval-augmented generation (RAG). However,
existing retrieval-augmented approaches face challenges in addressing diverse
queries and documents, particularly for medical knowledge queries, resulting in
sub-optimal performance. To address these limitations, we propose a novel
plug-and-play LLM-based retrieval method called Self-Rewarding Tree Search
(SeRTS) based on Monte Carlo Tree Search (MCTS) and a self-rewarding paradigm.
By combining the reasoning capabilities of LLMs with the effectiveness of tree
search, SeRTS boosts the zero-shot performance of retrieving high-quality and
informative results for RAG. We further enhance retrieval performance by
fine-tuning LLMs with Proximal Policy Optimization (PPO) objectives using the
trajectories collected by SeRTS as feedback. Controlled experiments using the
BioASQ-QA dataset with GPT-3.5-Turbo and LLama2-7b demonstrate that our method
significantly improves the performance of the BM25 retriever and surpasses the
strong baseline of self-reflection in both efficiency and scalability.
Moreover, SeRTS generates higher-quality feedback for PPO training than
self-reflection. Our proposed method effectively adapts LLMs to document
retrieval tasks, enhancing their ability to retrieve highly relevant documents
for RAG in the context of medical knowledge queries. This work presents a
significant step forward in leveraging LLMs for accurate and comprehensive
biomedical question answering.
