  Video Super-Resolution (VSR) aims to recover sequences of high-resolution
(HR) frames from low-resolution (LR) frames. Previous methods mainly utilize
temporally adjacent frames to assist the reconstruction of target frames.
However, in the real world, there is a lot of irrelevant information in
adjacent frames of videos with fast scene switching, these VSR methods cannot
adaptively distinguish and select useful information. In contrast, with a
transformer structure suitable for temporal tasks, we devise a novel adaptive
scenario video super-resolution method. Specifically, we use optical flow to
label the patches in each video frame, only calculate the attention of patches
with the same label. Then select the most relevant label among them to
supplement the spatial-temporal information of the target frame. This design
can directly make the supplementary information come from the same scene as
much as possible. We further propose a cross-scale feature aggregation module
to better handle the scale variation problem. Compared with other video
super-resolution methods, our method not only achieves significant performance
gains on single-scene videos but also has better robustness on cross-scene
datasets.
