  Solutions to math word problems (MWPs) with step-by-step explanations are
valuable, especially in education, to help students better comprehend
problem-solving strategies. Most existing approaches only focus on obtaining
the final correct answer. A few recent approaches leverage intermediate
solution steps to improve final answer correctness but often cannot generate
coherent steps with a clear solution strategy. Contrary to existing work, we
focus on improving the correctness and coherence of the intermediate solutions
steps. We propose a step-by-step planning approach for intermediate solution
generation, which strategically plans the generation of the next solution step
based on the MWP and the previous solution steps. Our approach first plans the
next step by predicting the necessary math operation needed to proceed, given
history steps, then generates the next step, token-by-token, by prompting a
language model with the predicted math operation. Experiments on the GSM8K
dataset demonstrate that our approach improves the accuracy and
interpretability of the solution on both automatic metrics and human
evaluation.
