  This paper introduces the MCT Self-Refine (MCTSr) algorithm, an innovative
integration of Large Language Models (LLMs) with Monte Carlo Tree Search
(MCTS), designed to enhance performance in complex mathematical reasoning
tasks. Addressing the challenges of accuracy and reliability in LLMs,
particularly in strategic and mathematical reasoning, MCTSr leverages
systematic exploration and heuristic self-refine mechanisms to improve
decision-making frameworks within LLMs. The algorithm constructs a Monte Carlo
search tree through iterative processes of Selection, self-refine,
self-evaluation, and Backpropagation, utilizing an improved Upper Confidence
Bound (UCB) formula to optimize the exploration-exploitation balance. Extensive
experiments demonstrate MCTSr's efficacy in solving Olympiad-level mathematical
problems, significantly improving success rates across multiple datasets,
including GSM8K, GSM Hard, MATH, and Olympiad-level benchmarks, including Math
Odyssey, AIME, and OlympiadBench. The study advances the application of LLMs in
complex reasoning tasks and sets a foundation for future AI integration,
enhancing decision-making accuracy and reliability in LLM-driven applications.
