  While forward reasoning (i.e., find the answer given the question) has been
explored extensively in recent literature, backward reasoning is relatively
unexplored. We examine the backward reasoning capabilities of LLMs on Math Word
Problems (MWPs): given a mathematical question and its answer, with some
details omitted from the question, can LLMs effectively retrieve the missing
information? On modifying three benchmark datasets for this task, to evaluate
this task: GSM8k, SVAMP, and MultiArith, we find a significant drop in the
accuracy of models on this task compared to forward reasoning across SOTA LLMs
(GPT4, GPT3.5, PaLM-2, and LLaMa). Motivated by the fact backward reasoning can
be seen as the ''inverse'' of forward reasoning, we propose variations of three
different forward reasoning strategies to improve performance. Rephrase
reformulates the given problem into a forward reasoning problem, PAL-Tools
combines the idea of Program-Aided LLMs to produce a set of equations that can
be solved by an external solver, and Check your Work exploits the availability
of natural verifier of high accuracy in the forward direction, interleaving
solving and verification steps. Finally, realizing that each of our base
methods correctly solves a different set of problems, we propose a novel
Bayesian formulation for creating an ensemble over the base methods to further
boost the accuracy. Extensive experimentation demonstrates successive
improvement in the performance of LLMs on the backward reasoning task, using
our strategies, with our ensemble-based method resulting in significant
performance gains compared to the SOTA forward reasoning strategies we adapt.
