  GANs are able to perform generation and manipulation tasks, trained on a
single video. However, these single video GANs require unreasonable amount of
time to train on a single video, rendering them almost impractical. In this
paper we question the necessity of a GAN for generation from a single video,
and introduce a non-parametric baseline for a variety of generation and
manipulation tasks. We revive classical space-time patches-nearest-neighbors
approaches and adapt them to a scalable unconditional generative model, without
any learning. This simple baseline surprisingly outperforms single-video GANs
in visual quality and realism (confirmed by quantitative and qualitative
evaluations), and is disproportionately faster (runtime reduced from several
days to seconds). Other than diverse video generation, we demonstrate other
applications using the same framework, including video analogies and
spatio-temporal retargeting. Our proposed approach is easily scaled to Full-HD
videos. These observations show that the classical approaches, if adapted
correctly, significantly outperform heavy deep learning machinery for these
tasks. This sets a new baseline for single-video generation and manipulation
tasks, and no less important -- makes diverse generation from a single video
practically possible for the first time.
