  Math word problem (MWP) is a challenging and critical task in natural
language processing. Many recent studies formalize MWP as a generation task and
have adopted sequence-to-sequence models to transform problem descriptions to
mathematical expressions. However, mathematical expressions are prone to minor
mistakes while the generation objective does not explicitly handle such
mistakes. To address this limitation, we devise a new ranking task for MWP and
propose Generate & Rank, a multi-task framework based on a generative
pre-trained language model. By joint training with generation and ranking, the
model learns from its own mistakes and is able to distinguish between correct
and incorrect expressions. Meanwhile, we perform tree-based disturbance
specially designed for MWP and an online update to boost the ranker. We
demonstrate the effectiveness of our proposed method on the benchmark and the
results show that our method consistently outperforms baselines in all
datasets. Particularly, in the classical Math23k, our method is 7% (78.4%
$\rightarrow$ 85.4%) higher than the state-of-the-art.
