  Humans move their hands and bodies together to communicate and solve tasks.
Capturing and replicating such coordinated activity is critical for virtual
characters that behave realistically. Surprisingly, most methods treat the 3D
modeling and tracking of bodies and hands separately. Here we formulate a model
of hands and bodies interacting together and fit it to full-body 4D sequences.
When scanning or capturing the full body in 3D, hands are small and often
partially occluded, making their shape and pose hard to recover. To cope with
low-resolution, occlusion, and noise, we develop a new model called MANO (hand
Model with Articulated and Non-rigid defOrmations). MANO is learned from around
1000 high-resolution 3D scans of hands of 31 subjects in a wide variety of hand
poses. The model is realistic, low-dimensional, captures non-rigid shape
changes with pose, is compatible with standard graphics packages, and can fit
any human hand. MANO provides a compact mapping from hand poses to pose blend
shape corrections and a linear manifold of pose synergies. We attach MANO to a
standard parameterized 3D body shape model (SMPL), resulting in a fully
articulated body and hand model (SMPL+H). We illustrate SMPL+H by fitting
complex, natural, activities of subjects captured with a 4D scanner. The
fitting is fully automatic and results in full body models that move naturally
with detailed hand motions and a realism not seen before in full body
performance capture. The models and data are freely available for research
purposes in our website (http://mano.is.tue.mpg.de).
