  In this work we propose a novel task framework under which a variety of
physical reasoning puzzles can be constructed using very simple rules. Under
sparse reward settings, most of these tasks can be very challenging for a
reinforcement learning agent to learn. We build several simple environments
with this task framework in Mujoco and OpenAI gym and attempt to solve them. We
are able to solve the environments by designing curricula to guide the agent in
learning and using imitation learning methods to transfer knowledge from a
simpler environment. This is only a first step for the task framework, and
further research on how to solve the harder tasks and transfer knowledge
between tasks is needed.
