  Understanding how humans interact with the world necessitates accurate 3D
hand pose estimation, a task complicated by the hand's high degree of
articulation, frequent occlusions, self-occlusions, and rapid motions. While
most existing methods rely on single-image inputs, videos have useful cues to
address aforementioned issues. However, existing video-based 3D hand datasets
are insufficient for training feedforward models to generalize to in-the-wild
scenarios. On the other hand, we have access to large human motion capture
datasets which also include hand motions, e.g. AMASS. Therefore, we develop a
generative motion prior specific for hands, trained on the AMASS dataset which
features diverse and high-quality hand motions. This motion prior is then
employed for video-based 3D hand motion estimation following a latent
optimization approach. Our integration of a robust motion prior significantly
enhances performance, especially in occluded scenarios. It produces stable,
temporally consistent results that surpass conventional single-frame methods.
We demonstrate our method's efficacy via qualitative and quantitative
evaluations on the HO3D and DexYCB datasets, with special emphasis on an
occlusion-focused subset of HO3D. Code is available at
https://hmp.is.tue.mpg.de
