  Emerging Metaverse applications demand accessible, accurate, and easy-to-use
tools for 3D digital human creations in order to depict different cultures and
societies as if in the physical world. Recent large-scale vision-language
advances pave the way to for novices to conveniently customize 3D content.
However, the generated CG-friendly assets still cannot represent the desired
facial traits for human characteristics. In this paper, we present DreamFace, a
progressive scheme to generate personalized 3D faces under text guidance. It
enables layman users to naturally customize 3D facial assets that are
compatible with CG pipelines, with desired shapes, textures, and fine-grained
animation capabilities. From a text input to describe the facial traits, we
first introduce a coarse-to-fine scheme to generate the neutral facial geometry
with a unified topology. We employ a selection strategy in the CLIP embedding
space, and subsequently optimize both the details displacements and normals
using Score Distillation Sampling from generic Latent Diffusion Model. Then,
for neutral appearance generation, we introduce a dual-path mechanism, which
combines the generic LDM with a novel texture LDM to ensure both the diversity
and textural specification in the UV space. We also employ a two-stage
optimization to perform SDS in both the latent and image spaces to
significantly provides compact priors for fine-grained synthesis. Our generated
neutral assets naturally support blendshapes-based facial animations. We
further improve the animation ability with personalized deformation
characteristics by learning the universal expression prior using the
cross-identity hypernetwork. Notably, DreamFace can generate of realistic 3D
facial assets with physically-based rendering quality and rich animation
ability from video footage, even for fashion icons or exotic characters in
cartoons and fiction movies.
