  Capitalizing on the recent advances in image generation models, existing
controllable face image synthesis methods are able to generate high-fidelity
images with some levels of controllability, e.g., controlling the shapes,
expressions, textures, and poses of the generated face images. However,
previous methods focus on controllable 2D image generative models, which are
prone to producing inconsistent face images under large expression and pose
changes. In this paper, we propose a new NeRF-based conditional 3D face
synthesis framework, which enables 3D controllability over the generated face
images by imposing explicit 3D conditions from 3D face priors. At its core is a
conditional Generative Occupancy Field (cGOF++) that effectively enforces the
shape of the generated face to conform to a given 3D Morphable Model (3DMM)
mesh, built on top of EG3D [1], a recent tri-plane-based generative model. To
achieve accurate control over fine-grained 3D face shapes of the synthesized
images, we additionally incorporate a 3D landmark loss as well as a volume
warping loss into our synthesis framework. Experiments validate the
effectiveness of the proposed method, which is able to generate high-fidelity
face images and shows more precise 3D controllability than state-of-the-art
2D-based controllable face synthesis methods.
