  Text-based games (TBGs) have become a popular proving ground for the
demonstration of learning-based agents that make decisions in quasi real-world
settings. The crux of the problem for a reinforcement learning agent in such
TBGs is identifying the objects in the world, and those objects' relations with
that world. While the recent use of text-based resources for increasing an
agent's knowledge and improving its generalization have shown promise, we posit
in this paper that there is much yet to be learned from visual representations
of these same worlds. Specifically, we propose to retrieve images that
represent specific instances of text observations from the world and train our
agents on such images. This improves the agent's overall understanding of the
game 'scene' and objects' relationships to the world around them, and the
variety of visual representations on offer allow the agent to generate a better
generalization of a relationship. We show that incorporating such images
improves the performance of agents in various TBG settings.
