  Large language models (LLMs) have been explored in a variety of reasoning
tasks including solving of mathematical problems. Each math dataset typically
includes its own specially designed evaluation script, which, while suitable
for its intended use, lacks generalizability across different datasets.
Consequently, updates and adaptations to these evaluation tools tend to occur
without being systematically reported, leading to inconsistencies and obstacles
to fair comparison across studies. To bridge this gap, we introduce a
comprehensive mathematical evaluation toolkit that not only utilizes a python
computer algebra system (CAS) for its numerical accuracy, but also integrates
an optional LLM, known for its considerable natural language processing
capabilities. To validate the effectiveness of our toolkit, we manually
annotated two distinct datasets. Our experiments demonstrate that the toolkit
yields more robust evaluation results compared to prior works, even without an
LLM. Furthermore, when an LLM is incorporated, there is a notable enhancement.
The code for our method will be made available at
\url{https://github.com/MARIO-Math-Reasoning/math_evaluation}.
