  We propose a novel method for generating high-resolution videos of
talking-heads from speech audio and a single 'identity' image. Our method is
based on a convolutional neural network model that incorporates a pre-trained
StyleGAN generator. We model each frame as a point in the latent space of
StyleGAN so that a video corresponds to a trajectory through the latent space.
Training the network is in two stages. The first stage is to model trajectories
in the latent space conditioned on speech utterances. To do this, we use an
existing encoder to invert the generator, mapping from each video frame into
the latent space. We train a recurrent neural network to map from speech
utterances to displacements in the latent space of the image generator. These
displacements are relative to the back-projection into the latent space of an
identity image chosen from the individuals depicted in the training dataset. In
the second stage, we improve the visual quality of the generated videos by
tuning the image generator on a single image or a short video of any chosen
identity. We evaluate our model on standard measures (PSNR, SSIM, FID and LMD)
and show that it significantly outperforms recent state-of-the-art methods on
one of two commonly used datasets and gives comparable performance on the
other. Finally, we report on ablation experiments that validate the components
of the model. The code and videos from experiments can be found at
https://mohammedalghamdi.github.io/talking-heads-acm-mm
