  We convert the DeepMind Mathematics Dataset into a reinforcement learning
environment by interpreting it as a program synthesis problem. Each action
taken in the environment adds an operator or an input into a discrete compute
graph. Graphs which compute correct answers yield positive reward, enabling the
optimization of a policy to construct compute graphs conditioned on problem
statements. Baseline models are trained using Double DQN on various subsets of
problem types, demonstrating the capability to learn to correctly construct
graphs despite the challenges of combinatorial explosion and noisy rewards.
