  We introduce DiffPhy, a differentiable physics-based model for articulated 3d
human motion reconstruction from video. Applications of physics-based reasoning
in human motion analysis have so far been limited, both by the complexity of
constructing adequate physical models of articulated human motion, and by the
formidable challenges of performing stable and efficient inference with physics
in the loop. We jointly address such modeling and inference challenges by
proposing an approach that combines a physically plausible body representation
with anatomical joint limits, a differentiable physics simulator, and
optimization techniques that ensure good performance and robustness to
suboptimal local optima. In contrast to several recent methods, our approach
readily supports full-body contact including interactions with objects in the
scene. Most importantly, our model connects end-to-end with images, thus
supporting direct gradient-based physics optimization by means of image-based
loss functions. We validate the model by demonstrating that it can accurately
reconstruct physically plausible 3d human motion from monocular video, both on
public benchmarks with available 3d ground-truth, and on videos from the
internet.
