  Hands are the main medium when people interact with the world. Generating
proper 3D motion for hand-object interaction is vital for applications such as
virtual reality and robotics. Although grasp tracking or object manipulation
synthesis can produce coarse hand motion, this kind of motion is inevitably
noisy and full of jitter. To address this problem, we propose a data-driven
method for coarse motion refinement. First, we design a hand-centric
representation to describe the dynamic spatial-temporal relation between hands
and objects. Compared to the object-centric representation, our hand-centric
representation is straightforward and does not require an ambiguous projection
process that converts object-based prediction into hand motion. Second, to
capture the dynamic clues of hand-object interaction, we propose a new
architecture that models the spatial and temporal structure in a hierarchical
manner. Extensive experiments demonstrate that our method outperforms previous
methods by a noticeable margin.
