  Obtaining policies that can generalise to new environments in reinforcement
learning is challenging. In this work, we demonstrate that language
understanding via a reading policy learner is a promising vehicle for
generalisation to new environments. We propose a grounded policy learning
problem, Read to Fight Monsters (RTFM), in which the agent must jointly reason
over a language goal, relevant dynamics described in a document, and
environment observations. We procedurally generate environment dynamics and
corresponding language descriptions of the dynamics, such that agents must read
to understand new environment dynamics instead of memorising any particular
information. In addition, we propose txt2$\pi$, a model that captures three-way
interactions between the goal, document, and observations. On RTFM, txt2$\pi$
generalises to new environments with dynamics not seen during training via
reading. Furthermore, our model outperforms baselines such as FiLM and
language-conditioned CNNs on RTFM. Through curriculum learning, txt2$\pi$
produces policies that excel on complex RTFM tasks requiring several reasoning
and coreference steps.
