  The combination of deep learning, artist-curated scans, and Implicit
Functions (IF), is enabling the creation of detailed, clothed, 3D humans from
images. However, existing methods are far from perfect. IF-based methods
recover free-form geometry, but produce disembodied limbs or degenerate shapes
for novel poses or clothes. To increase robustness for these cases, existing
work uses an explicit parametric body model to constrain surface
reconstruction, but this limits the recovery of free-form surfaces such as
loose clothing that deviates from the body. What we want is a method that
combines the best properties of implicit representation and explicit body
regularization. To this end, we make two key observations: (1) current networks
are better at inferring detailed 2D maps than full-3D surfaces, and (2) a
parametric model can be seen as a "canvas" for stitching together detailed
surface patches. Based on these, our method, ECON, has three main steps: (1) It
infers detailed 2D normal maps for the front and back side of a clothed person.
(2) From these, it recovers 2.5D front and back surfaces, called d-BiNI, that
are equally detailed, yet incomplete, and registers these w.r.t. each other
with the help of a SMPL-X body mesh recovered from the image. (3) It "inpaints"
the missing geometry between d-BiNI surfaces. If the face and hands are noisy,
they can optionally be replaced with the ones of SMPL-X. As a result, ECON
infers high-fidelity 3D humans even in loose clothes and challenging poses.
This goes beyond previous methods, according to the quantitative evaluation on
the CAPE and Renderpeople datasets. Perceptual studies also show that ECON's
perceived realism is better by a large margin. Code and models are available
for research purposes at econ.is.tue.mpg.de
