  Understanding and interacting with everyday physical scenes requires rich
knowledge about the structure of the world, represented either implicitly in a
value or policy function, or explicitly in a transition model. Here we
introduce a new class of learnable models--based on graph networks--which
implement an inductive bias for object- and relation-centric representations of
complex, dynamical systems. Our results show that as a forward model, our
approach supports accurate predictions from real and simulated data, and
surprisingly strong and efficient generalization, across eight distinct
physical systems which we varied parametrically and structurally. We also found
that our inference model can perform system identification. Our models are also
differentiable, and support online planning via gradient-based trajectory
optimization, as well as offline policy optimization. Our framework offers new
opportunities for harnessing and exploiting rich knowledge about the world, and
takes a key step toward building machines with more human-like representations
of the world.
