  Many computer vision tasks rely on labeled data. Rapid progress in generative
modeling has led to the ability to synthesize photorealistic images. However,
controlling specific aspects of the generation process such that the data can
be used for supervision of downstream tasks remains challenging. In this paper
we propose a novel generative model for images of faces, that is capable of
producing high-quality images under fine-grained control over eye gaze and head
orientation angles. This requires the disentangling of many appearance related
factors including gaze and head orientation but also lighting, hue etc. We
propose a novel architecture which learns to discover, disentangle and encode
these extraneous variations in a self-learned manner. We further show that
explicitly disentangling task-irrelevant factors results in more accurate
modelling of gaze and head orientation. A novel evaluation scheme shows that
our method improves upon the state-of-the-art in redirection accuracy and
disentanglement between gaze direction and head orientation changes.
Furthermore, we show that in the presence of limited amounts of real-world
training data, our method allows for improvements in the downstream task of
semi-supervised cross-dataset gaze estimation. Please check our project page
at: https://ait.ethz.ch/projects/2020/STED-gaze/
