  Accurate 3D reconstruction of the hand and object shape from a hand-object
image is important for understanding human-object interaction as well as human
daily activities. Different from bare hand pose estimation, hand-object
interaction poses a strong constraint on both the hand and its manipulated
object, which suggests that hand configuration may be crucial contextual
information for the object, and vice versa. However, current approaches address
this task by training a two-branch network to reconstruct the hand and object
separately with little communication between the two branches. In this work, we
propose to consider hand and object jointly in feature space and explore the
reciprocity of the two branches. We extensively investigate cross-branch
feature fusion architectures with MLP or LSTM units. Among the investigated
architectures, a variant with LSTM units that enhances object feature with hand
feature shows the best performance gain. Moreover, we employ an auxiliary depth
estimation module to augment the input RGB image with the estimated depth map,
which further improves the reconstruction accuracy. Experiments conducted on
public datasets demonstrate that our approach significantly outperforms
existing approaches in terms of the reconstruction accuracy of objects.
