  In this paper, we consider the task of space-time video super-resolution
(ST-VSR), namely, expanding a given source video to a higher frame rate and
resolution simultaneously. However, most existing schemes either consider a
fixed intermediate time and scale in the training stage or only accept a preset
number of input frames (e.g., two adjacent frames) that fails to exploit
long-range temporal information. To address these problems, we propose a
continuous ST-VSR (C-STVSR) method that can convert the given video to any
frame rate and spatial resolution. To achieve time-arbitrary interpolation, we
propose a forward warping guided frame synthesis module and an
optical-flow-guided context consistency loss to better approximate extreme
motion and preserve similar structures among input and prediction frames. In
addition, we design a memory-friendly cascading depth-to-space module to
realize continuous spatial upsampling. Meanwhile, with the sophisticated
reorganization of optical flow, the proposed method is memory friendly, making
it possible to propagate information from long-range neighboring frames and
achieve better reconstruction quality. Extensive experiments show that the
proposed algorithm has good flexibility and achieves better performance on
various datasets compared with the state-of-the-art methods in both objective
evaluations and subjective visual effects.
