  We present the Neural Physics Engine (NPE), a framework for learning
simulators of intuitive physics that naturally generalize across variable
object count and different scene configurations. We propose a factorization of
a physical scene into composable object-based representations and a neural
network architecture whose compositional structure factorizes object dynamics
into pairwise interactions. Like a symbolic physics engine, the NPE is endowed
with generic notions of objects and their interactions; realized as a neural
network, it can be trained via stochastic gradient descent to adapt to specific
object properties and dynamics of different worlds. We evaluate the efficacy of
our approach on simple rigid body dynamics in two-dimensional worlds. By
comparing to less structured architectures, we show that the NPE's
compositional representation of the structure in physical interactions improves
its ability to predict movement, generalize across variable object count and
different scene configurations, and infer latent properties of objects such as
mass.
