  We present a new trainable system for physically plausible markerless 3D
human motion capture, which achieves state-of-the-art results in a broad range
of challenging scenarios. Unlike most neural methods for human motion capture,
our approach, which we dub physionical, is aware of physical and environmental
constraints. It combines in a fully differentiable way several key innovations,
i.e., 1. a proportional-derivative controller, with gains predicted by a neural
network, that reduces delays even in the presence of fast motions, 2. an
explicit rigid body dynamics model and 3. a novel optimisation layer that
prevents physically implausible foot-floor penetration as a hard constraint.
The inputs to our system are 2D joint keypoints, which are canonicalised in a
novel way so as to reduce the dependency on intrinsic camera parameters -- both
at train and test time. This enables more accurate global translation
estimation without generalisability loss. Our model can be finetuned only with
2D annotations when the 3D annotations are not available. It produces smooth
and physically principled 3D motions in an interactive frame rate in a wide
variety of challenging scenes, including newly recorded ones. Its advantages
are especially noticeable on in-the-wild sequences that significantly differ
from common 3D pose estimation benchmarks such as Human 3.6M and MPI-INF-3DHP.
Qualitative results are available at
http://gvv.mpi-inf.mpg.de/projects/PhysAware/
