  Accurately modeling detailed interactions between human/hand and object is an
appealing yet challenging task. Current multi-view capture systems are only
capable of reconstructing multiple subjects into a single, unified mesh, which
fails to model the states of each instance individually during interactions. To
address this, previous methods use template-based representations to track
human/hand and object. However, the quality of the reconstructions is limited
by the descriptive capabilities of the templates so that these methods are
inherently struggle with geometry details, pressing deformations and invisible
contact surfaces. In this work, we propose an end-to-end Instance-aware
Human-Object Interactions recovery (Ins-HOI) framework by introducing an
instance-level occupancy field representation. However, the real-captured data
is presented as a holistic mesh, unable to provide instance-level supervision.
To address this, we further propose a complementary training strategy that
leverages synthetic data to introduce instance-level shape priors, enabling the
disentanglement of occupancy fields for different instances. Specifically,
synthetic data, created by randomly combining individual scans of humans/hands
and objects, guides the network to learn a coarse prior of instances.
Meanwhile, real-captured data helps in learning the overall geometry and
restricting interpenetration in contact areas. As demonstrated in experiments,
our method Ins-HOI supports instance-level reconstruction and provides
reasonable and realistic invisible contact surfaces even in cases of extremely
close interaction. To facilitate the research of this task, we collect a
large-scale, high-fidelity 3D scan dataset, including 5.2k high-quality scans
with real-world human-chair and hand-object interactions. The code and data
will be public for research purposes.
