  Garment sewing pattern represents the intrinsic rest shape of a garment, and
is the core for many applications like fashion design, virtual try-on, and
digital avatars. In this work, we explore the challenging problem of recovering
garment sewing patterns from daily photos for augmenting these applications. To
solve the problem, we first synthesize a versatile dataset, named SewFactory,
which consists of around 1M images and ground-truth sewing patterns for model
training and quantitative evaluation. SewFactory covers a wide range of human
poses, body shapes, and sewing patterns, and possesses realistic appearances
thanks to the proposed human texture synthesis network. Then, we propose a
two-level Transformer network called Sewformer, which significantly improves
the sewing pattern prediction performance. Extensive experiments demonstrate
that the proposed framework is effective in recovering sewing patterns and well
generalizes to casually-taken human photos. Code, dataset, and pre-trained
models are available at: https://sewformer.github.io.
