  We propose a novel end-to-end semi-supervised adversarial framework to
generate photorealistic face images of new identities with wide ranges of
expressions, poses, and illuminations conditioned by a 3D morphable model.
Previous adversarial style-transfer methods either supervise their networks
with large volume of paired data or use unpaired data with a highly
under-constrained two-way generative framework in an unsupervised fashion. We
introduce pairwise adversarial supervision to constrain two-way domain
adaptation by a small number of paired real and synthetic images for training
along with the large volume of unpaired data. Extensive qualitative and
quantitative experiments are performed to validate our idea. Generated face
images of new identities contain pose, lighting and expression diversity and
qualitative results show that they are highly constraint by the synthetic input
image while adding photorealism and retaining identity information. We combine
face images generated by the proposed method with the real data set to train
face recognition algorithms. We evaluated the model on two challenging data
sets: LFW and IJB-A. We observe that the generated images from our framework
consistently improves over the performance of deep face recognition network
trained with Oxford VGG Face dataset and achieves comparable results to the
state-of-the-art.
