  Recent advances in 3D-aware generative models (3D-aware GANs) combined with
Neural Radiance Fields (NeRF) have achieved impressive results. However no
prior works investigate 3D-aware GANs for 3D consistent multi-class
image-to-image (3D-aware I2I) translation. Naively using 2D-I2I translation
methods suffers from unrealistic shape/identity change. To perform 3D-aware
multi-class I2I translation, we decouple this learning process into a
multi-class 3D-aware GAN step and a 3D-aware I2I translation step. In the first
step, we propose two novel techniques: a new conditional architecture and an
effective training strategy. In the second step, based on the well-trained
multi-class 3D-aware GAN architecture, that preserves view-consistency, we
construct a 3D-aware I2I translation system. To further reduce the
view-consistency problems, we propose several new techniques, including a
U-net-like adaptor network design, a hierarchical representation constrain and
a relative regularization loss. In extensive experiments on two datasets,
quantitative and qualitative results demonstrate that we successfully perform
3D-aware I2I translation with multi-view consistency.
