  We introduce a new silhouette-based representation for modeling clothed human
bodies using deep generative models. Our method can reconstruct a complete and
textured 3D model of a person wearing clothes from a single input picture.
Inspired by the visual hull algorithm, our implicit representation uses 2D
silhouettes and 3D joints of a body pose to describe the immense shape
complexity and variations of clothed people. Given a segmented 2D silhouette of
a person and its inferred 3D joints from the input picture, we first synthesize
consistent silhouettes from novel view points around the subject. The
synthesized silhouettes which are the most consistent with the input
segmentation are fed into a deep visual hull algorithm for robust 3D shape
prediction. We then infer the texture of the subject's back view using the
frontal image and segmentation mask as input to a conditional generative
adversarial network. Our experiments demonstrate that our silhouette-based
model is an effective representation and the appearance of the back view can be
predicted reliably using an image-to-image translation network. While classic
methods based on parametric models often fail for single-view images of
subjects with challenging clothing, our approach can still produce successful
results, which are comparable to those obtained from multi-view input.
