Schema Networks: Zero-shot Transfer with a Generative Causal Model of
  Intuitive Physics
  The recent adaptation of deep neural network-based methods to reinforcement
learning and planning domains has yielded remarkable progress on individual
tasks. Nonetheless, progress on task-to-task transfer remains limited. In
pursuit of efficient and robust generalization, we introduce the Schema
Network, an object-oriented generative physics simulator capable of
disentangling multiple causes of events and reasoning backward through causes
to achieve goals. The richly structured architecture of the Schema Network can
learn the dynamics of an environment directly from data. We compare Schema
Networks with Asynchronous Advantage Actor-Critic and Progressive Networks on a
suite of Breakout variations, reporting results on training efficiency and
zero-shot generalization, consistently demonstrating faster, more robust
learning and better transfer. We argue that generalizing from limited data and
learning causal relationships are essential abilities on the path toward
generally intelligent systems.
