  To solve tasks in new environments involving objects unseen during training,
agents must reason over prior information about those objects and their
relations. We introduce the Prior Knowledge Graph network, an architecture for
combining prior information, structured as a knowledge graph, with a symbolic
parsing of the visual scene, and demonstrate that this approach is able to
apply learned relations to novel objects whereas the baseline algorithms fail.
Ablation experiments show that the agents ground the knowledge graph relations
to semantically-relevant behaviors. In both a Sokoban game and the more complex
Pacman environment, our network is also more sample efficient than the
baselines, reaching the same performance in 5-10x fewer episodes. Once the
agents are trained with our approach, we can manipulate agent behavior by
modifying the knowledge graph in semantically meaningful ways. These results
suggest that our network provides a framework for agents to reason over
structured knowledge graphs while still leveraging gradient based learning
approaches.
