  In this paper, we aim to create generalizable and controllable neural signed
distance fields (SDFs) that represent clothed humans from monocular depth
observations. Recent advances in deep learning, especially neural implicit
representations, have enabled human shape reconstruction and controllable
avatar generation from different sensor inputs. However, to generate realistic
cloth deformations from novel input poses, watertight meshes or dense full-body
scans are usually needed as inputs. Furthermore, due to the difficulty of
effectively modeling pose-dependent cloth deformations for diverse body shapes
and cloth types, existing approaches resort to per-subject/cloth-type
optimization from scratch, which is computationally expensive. In contrast, we
propose an approach that can quickly generate realistic clothed human avatars,
represented as controllable neural SDFs, given only monocular depth images. We
achieve this by using meta-learning to learn an initialization of a
hypernetwork that predicts the parameters of neural SDFs. The hypernetwork is
conditioned on human poses and represents a clothed neural avatar that deforms
non-rigidly according to the input poses. Meanwhile, it is meta-learned to
effectively incorporate priors of diverse body shapes and cloth types and thus
can be much faster to fine-tune, compared to models trained from scratch. We
qualitatively and quantitatively show that our approach outperforms
state-of-the-art approaches that require complete meshes as inputs while our
approach requires only depth frames as inputs and runs orders of magnitudes
faster. Furthermore, we demonstrate that our meta-learned hypernetwork is very
robust, being the first to generate avatars with realistic dynamic cloth
deformations given as few as 8 monocular depth frames.
