  Researchers have explored various ways to generate realistic images from
freehand sketches, e.g., for objects and human faces. However, how to generate
realistic human body images from sketches is still a challenging problem. It
is, first because of the sensitivity to human shapes, second because of the
complexity of human images caused by body shape and pose changes, and third
because of the domain gap between realistic images and freehand sketches. In
this work, we present DeepPortraitDrawing, a deep generative framework for
converting roughly drawn sketches to realistic human body images. To encode
complicated body shapes under various poses, we take a local-to-global
approach. Locally, we employ semantic part auto-encoders to construct
part-level shape spaces, which are useful for refining the geometry of an input
pre-segmented hand-drawn sketch. Globally, we employ a cascaded spatial
transformer network to refine the structure of body parts by adjusting their
spatial locations and relative proportions. Finally, we use a global synthesis
network for the sketch-to-image translation task, and a face refinement network
to enhance facial details. Extensive experiments have shown that given roughly
sketched human portraits, our method produces more realistic images than the
state-of-the-art sketch-to-image synthesis techniques.
