  We propose an approach to estimate arm and hand dynamics from monocular video
by utilizing the relationship between arm and hand. Although monocular full
human motion capture technologies have made great progress in recent years,
recovering accurate and plausible arm twists and hand gestures from in-the-wild
videos still remains a challenge. To solve this problem, our solution is
proposed based on the fact that arm poses and hand gestures are highly
correlated in most real situations. To fully exploit arm-hand correlation as
well as inter-frame information, we carefully design a Spatial-Temporal
Parallel Arm-Hand Motion Transformer (PAHMT) to predict the arm and hand
dynamics simultaneously. We also introduce new losses to encourage the
estimations to be smooth and accurate. Besides, we collect a motion capture
dataset including 200K frames of hand gestures and use this data to train our
model. By integrating a 2D hand pose estimation model and a 3D human pose
estimation model, the proposed method can produce plausible arm and hand
dynamics from monocular video. Extensive evaluations demonstrate that the
proposed method has advantages over previous state-of-the-art approaches and
shows robustness under various challenging scenarios.
