  3D hand-object pose estimation is the key to the success of many computer
vision applications. The main focus of this task is to effectively model the
interaction between the hand and an object. To this end, existing works either
rely on interaction constraints in a computationally-expensive iterative
optimization, or consider only a sparse correlation between sampled hand and
object keypoints. In contrast, we propose a novel dense mutual attention
mechanism that is able to model fine-grained dependencies between the hand and
the object. Specifically, we first construct the hand and object graphs
according to their mesh structures. For each hand node, we aggregate features
from every object node by the learned attention and vice versa for each object
node. Thanks to such dense mutual attention, our method is able to produce
physically plausible poses with high quality and real-time inference speed.
Extensive quantitative and qualitative experiments on large benchmark datasets
show that our method outperforms state-of-the-art methods. The code is
available at https://github.com/rongakowang/DenseMutualAttention.git.
