  Differentiable simulators provide an avenue for closing the sim-to-real gap
by enabling the use of efficient, gradient-based optimization algorithms to
find the simulation parameters that best fit the observed sensor readings.
Nonetheless, these analytical models can only predict the dynamical behavior of
systems for which they have been designed. In this work, we study the
augmentation of a novel differentiable rigid-body physics engine via neural
networks that is able to learn nonlinear relationships between dynamic
quantities and can thus learn effects not accounted for in traditional
simulators.Such augmentations require less data to train and generalize better
compared to entirely data-driven models. Through extensive experiments, we
demonstrate the ability of our hybrid simulator to learn complex dynamics
involving frictional contacts from real data, as well as match known models of
viscous friction, and present an approach for automatically discovering useful
augmentations. We show that, besides benefiting dynamics modeling, inserting
neural networks can accelerate model-based control architectures. We observe a
ten-fold speed-up when replacing the QP solver inside a model-predictive gait
controller for quadruped robots with a neural network, allowing us to
significantly improve control delays as we demonstrate in real-hardware
experiments.
  We publish code, additional results and videos from our experiments on our
project webpage at https://sites.google.com/usc.edu/neuralsim.
