  Video super-resolution (VSR) is a task that aims to reconstruct
high-resolution (HR) frames from the low-resolution (LR) reference frame and
multiple neighboring frames. The vital operation is to utilize the relative
misaligned frames for the current frame reconstruction and preserve the
consistency of the results. Existing methods generally explore information
propagation and frame alignment to improve the performance of VSR. However, few
studies focus on the temporal consistency of inter-frames. In this paper, we
propose a Temporal Consistency learning Network (TCNet) for VSR in an
end-to-end manner, to enhance the consistency of the reconstructed videos. A
spatio-temporal stability module is designed to learn the self-alignment from
inter-frames. Especially, the correlative matching is employed to exploit the
spatial dependency from each frame to maintain structural stability. Moreover,
a self-attention mechanism is utilized to learn the temporal correspondence to
implement an adaptive warping operation for temporal consistency among
multi-frames. Besides, a hybrid recurrent architecture is designed to leverage
short-term and long-term information. We further present a progressive fusion
module to perform a multistage fusion of spatio-temporal features. And the
final reconstructed frames are refined by these fused features. Objective and
subjective results of various experiments demonstrate that TCNet has superior
performance on different benchmark datasets, compared to several
state-of-the-art methods.
