  Most prior works in perceiving 3D humans from images reason human in
isolation without their surroundings. However, humans are constantly
interacting with the surrounding objects, thus calling for models that can
reason about not only the human but also the object and their interaction. The
problem is extremely challenging due to heavy occlusions between humans and
objects, diverse interaction types and depth ambiguity. In this paper, we
introduce CHORE, a novel method that learns to jointly reconstruct the human
and the object from a single RGB image. CHORE takes inspiration from recent
advances in implicit surface learning and classical model-based fitting. We
compute a neural reconstruction of human and object represented implicitly with
two unsigned distance fields, a correspondence field to a parametric body and
an object pose field. This allows us to robustly fit a parametric body model
and a 3D object template, while reasoning about interactions. Furthermore,
prior pixel-aligned implicit learning methods use synthetic data and make
assumptions that are not met in the real data. We propose a elegant depth-aware
scaling that allows more efficient shape learning on real data. Experiments
show that our joint reconstruction learned with the proposed strategy
significantly outperforms the SOTA. Our code and models are available at
https://virtualhumans.mpi-inf.mpg.de/chore
