  The ability to quickly solve a wide range of real-world tasks requires a
commonsense understanding of the world. Yet, how to best extract such knowledge
from natural language corpora and integrate it with reinforcement learning (RL)
agents remains an open challenge. This is partly due to the lack of lightweight
simulation environments that sufficiently reflect the semantics of the real
world and provide knowledge sources grounded with respect to observations in an
RL environment. To better enable research on agents making use of commonsense
knowledge, we propose WordCraft, an RL environment based on Little Alchemy 2.
This lightweight environment is fast to run and built upon entities and
relations inspired by real-world semantics. We evaluate several representation
learning methods on this new benchmark and propose a new method for integrating
knowledge graphs with an RL agent.
