  Differentiable physics is a powerful tool in computer vision and robotics for
scene understanding and reasoning about interactions. Existing approaches have
frequently been limited to objects with simple shape or shapes that are known
in advance. In this paper, we propose a novel approach to differentiable
physics with frictional contacts which represents object shapes implicitly
using signed distance fields (SDFs). Our simulation supports contact point
calculation even when the involved shapes are nonconvex. Moreover, we propose
ways for differentiating the dynamics for the object shape to facilitate shape
optimization using gradient-based methods. In our experiments, we demonstrate
that our approach allows for model-based inference of physical parameters such
as friction coefficients, mass, forces or shape parameters from trajectory and
depth image observations in several challenging synthetic scenarios and a real
image sequence.
