  Interacting systems are prevalent in nature, from dynamical systems in
physics to complex societal dynamics. The interplay of components can give rise
to complex behavior, which can often be explained using a simple model of the
system's constituent parts. In this work, we introduce the neural relational
inference (NRI) model: an unsupervised model that learns to infer interactions
while simultaneously learning the dynamics purely from observational data. Our
model takes the form of a variational auto-encoder, in which the latent code
represents the underlying interaction graph and the reconstruction is based on
graph neural networks. In experiments on simulated physical systems, we show
that our NRI model can accurately recover ground-truth interactions in an
unsupervised manner. We further demonstrate that we can find an interpretable
structure and predict complex dynamics in real motion capture and sports
tracking data.
