  This paper presents a novel end-to-end dynamic time-lapse video generation
framework, named DTVNet, to generate diversified time-lapse videos from a
single landscape image conditioned on normalized motion vectors. The proposed
DTVNet consists of two submodules: \emph{Optical Flow Encoder} (OFE) and
\emph{Dynamic Video Generator} (DVG). The OFE maps a sequence of optical flow
maps to a \emph{normalized motion vector} that encodes the motion information
of the generated video. The DVG contains motion and content streams to learn
from the motion vector and the single landscape image. Besides, it contains an
encoder to learn shared content features and a decoder to construct video
frames with corresponding motion. Specifically, the \emph{motion stream}
introduces multiple \emph{adaptive instance normalization} (AdaIN) layers to
integrate multi-level motion information for controlling the object motion. In
the testing stage, videos with the same content but various motion information
can be generated by different \emph{normalized motion vectors} based on only
one input image. Also, we propose a high-resolution scenic time-lapse video
dataset, named Quick-Sky-Time, to evaluate different approaches, which can be
viewed as a new benchmark for high-quality scenic image and video generation
tasks. We further conduct experiments on Sky Time-lapse, Beach, and
Quick-Sky-Time datasets. The results demonstrate the superiority of our
approach over state-of-the-art methods for generating high-quality and various
dynamic videos.
