  This study presents a novel learning approach designed to enhance both
mathematical reasoning and problem-solving abilities of Large Language Models
(LLMs). We focus on integrating the Chain-of-Thought (CoT) and the
Program-of-Thought (PoT) learning, hypothesizing that prioritizing the learning
of mathematical reasoning ability is helpful for the amplification of
problem-solving ability. Thus, the initial learning with CoT is essential for
solving challenging mathematical problems. To this end, we propose a sequential
learning approach, named SAAS (Solving Ability Amplification Strategy), which
strategically transitions from CoT learning to PoT learning. Our empirical
study, involving an extensive performance comparison using several benchmarks,
demonstrates that our SAAS achieves state-of-the-art (SOTA) performance. The
results underscore the effectiveness of our sequential learning approach,
marking a significant advancement in the field of mathematical reasoning in
LLMs.
