  Intelligent agents need a physical understanding of the world to predict the
impact of their actions in the future. While learning-based models of the
environment dynamics have contributed to significant improvements in sample
efficiency compared to model-free reinforcement learning algorithms, they
typically fail to generalize to system states beyond the training data, while
often grounding their predictions on non-interpretable latent variables.
  We introduce Interactive Differentiable Simulation (IDS), a differentiable
physics engine, that allows for efficient, accurate inference of physical
properties of rigid-body systems. Integrated into deep learning architectures,
our model is able to accomplish system identification using visual input,
leading to an interpretable model of the world whose parameters have physical
meaning. We present experiments showing automatic task-based robot design and
parameter estimation for nonlinear dynamical systems by automatically
calculating gradients in IDS. When integrated into an adaptive model-predictive
control algorithm, our approach exhibits orders of magnitude improvements in
sample efficiency over model-free reinforcement learning algorithms on
challenging nonlinear control domains.
