  Enabling robots to perform complex dynamic tasks such as picking up an object
in one sweeping motion or pushing off a wall to quickly turn a corner is a
challenging problem. The dynamic interactions implicit in these tasks are
critical towards the successful execution of such tasks. Graph neural networks
(GNNs) provide a principled way of learning the dynamics of interactive systems
but can suffer from scaling issues as the number of interactions increases.
Furthermore, the problem of using learned GNN-based models for optimal control
is insufficiently explored. In this work, we present a method for efficiently
learning the dynamics of interacting systems by simultaneously learning a
dynamic graph structure and a stable and locally linear forward model of the
system. The dynamic graph structure encodes evolving contact modes along a
trajectory by making probabilistic predictions over the edges of the graph.
Additionally, we introduce a temporal dependence in the learned graph structure
which allows us to incorporate contact measurement updates during execution
thus enabling more accurate forward predictions. The learned stable and locally
linear dynamics enable the use of optimal control algorithms such as iLQR for
long-horizon planning and control for complex interactive tasks. Through
experiments in simulation and in the real world, we evaluate the performance of
our method by using the learned interaction dynamics for control and
demonstrate generalization to more objects and interactions not seen during
training. We introduce a control scheme that takes advantage of contact
measurement updates and hence is robust to prediction inaccuracies during
execution.
