  Modeling hand-object interactions is a fundamentally challenging task in 3D
computer vision. Despite remarkable progress that has been achieved in this
field, existing methods still fail to synthesize the hand-object interaction
photo-realistically, suffering from degraded rendering quality caused by the
heavy mutual occlusions between the hand and the object, and inaccurate
hand-object pose estimation. To tackle these challenges, we present a novel
free-viewpoint rendering framework, Neural Contact Radiance Field (NCRF), to
reconstruct hand-object interactions from a sparse set of videos. In
particular, the proposed NCRF framework consists of two key components: (a) A
contact optimization field that predicts an accurate contact field from 3D
query points for achieving desirable contact between the hand and the object.
(b) A hand-object neural radiance field to learn an implicit hand-object
representation in a static canonical space, in concert with the specifically
designed hand-object motion field to produce observation-to-canonical
correspondences. We jointly learn these key components where they mutually help
and regularize each other with visual and geometric constraints, producing a
high-quality hand-object reconstruction that achieves photo-realistic novel
view synthesis. Extensive experiments on HO3D and DexYCB datasets show that our
approach outperforms the current state-of-the-art in terms of both rendering
quality and pose estimation accuracy.
