  Generating photorealistic images of human faces at scale remains a
prohibitively difficult task using computer graphics approaches. This is
because these require the simulation of light to be photorealistic, which in
turn requires physically accurate modelling of geometry, materials, and light
sources, for both the head and the surrounding scene. Non-photorealistic
renders however are increasingly easy to produce. In contrast to computer
graphics approaches, generative models learned from more readily available 2D
image data have been shown to produce samples of human faces that are hard to
distinguish from real data. The process of learning usually corresponds to a
loss of control over the shape and appearance of the generated images. For
instance, even simple disentangling tasks such as modifying the hair
independently of the face, which is trivial to accomplish in a computer
graphics approach, remains an open research question. In this work, we propose
an algorithm that matches a non-photorealistic, synthetically generated image
to a latent vector of a pretrained StyleGAN2 model which, in turn, maps the
vector to a photorealistic image of a person of the same pose, expression,
hair, and lighting. In contrast to most previous work, we require no synthetic
training data. To the best of our knowledge, this is the first algorithm of its
kind to work at a resolution of 1K and represents a significant leap forward in
visual realism.
