  Modeling the hand-object (HO) interaction not only requires estimation of the
HO pose, but also pays attention to the contact due to their interaction.
Significant progress has been made in estimating hand and object separately
with deep learning methods, simultaneous HO pose estimation and contact
modeling has not yet been fully explored. In this paper, we present an explicit
contact representation namely Contact Potential Field (CPF), and a
learning-fitting hybrid framework namely MIHO to Modeling the Interaction of
Hand and Object. In CPF, we treat each contacting HO vertex pair as a
spring-mass system. Hence the whole system forms a potential field with minimal
elastic energy at the grasp position. Extensive experiments on the two commonly
used benchmarks have demonstrated that our method can achieve state-of-the-art
in several reconstruction metrics, and allow us to produce more physically
plausible HO pose even when the ground-truth exhibits severe interpenetration
or disjointedness. Our code is available at https://github.com/lixiny/CPF.
