  Due to the rapid advancements in multimodal large language models, evaluating
their multimodal mathematical capabilities continues to receive wide attention.
Despite the datasets like MathVista proposed benchmarks for assessing
mathematical capabilities in multimodal scenarios, there is still a lack of
corresponding evaluation tools and datasets for fine-grained assessment in the
context of K12 education in Chinese language. To systematically evaluate the
capability of multimodal large models in solving Chinese multimodal
mathematical problems, we propose a Chinese Multi-modal Math Skill Evaluation
Benchmark, named CMMaTH, contraining 23k multimodal K12 math related questions,
forming the largest Chinese multimodal mathematical problem benchmark to date.
CMMaTH questions from elementary to high school levels, provide increased
diversity in problem types, solution objectives, visual elements, detailed
knowledge points, and standard solution annotations. We have constructed an
open-source tool GradeGPT integrated with the CMMaTH dataset, facilitating
stable, rapid, and cost-free model evaluation. Our data and code are available.
