  Generative adversarial models (GANs) continue to produce advances in terms of
the visual quality of still images, as well as the learning of temporal
correlations. However, few works manage to combine these two interesting
capabilities for the synthesis of video content: Most methods require an
extensive training dataset to learn temporal correlations, while being rather
limited in the resolution and visual quality of their output. We present a
novel approach to the video synthesis problem that helps to greatly improve
visual quality and drastically reduce the amount of training data and resources
necessary for generating videos. Our formulation separates the spatial domain,
in which individual frames are synthesized, from the temporal domain, in which
motion is generated. For the spatial domain we use a pre-trained StyleGAN
network, the latent space of which allows control over the appearance of the
objects it was trained for. The expressive power of this model allows us to
embed our training videos in the StyleGAN latent space. Our temporal
architecture is then trained not on sequences of RGB frames, but on sequences
of StyleGAN latent codes. The advantageous properties of the StyleGAN space
simplify the discovery of temporal correlations. We demonstrate that it
suffices to train our temporal architecture on only 10 minutes of footage of 1
subject for about 6 hours. After training, our model can not only generate new
portrait videos for the training subject, but also for any random subject which
can be embedded in the StyleGAN space.
