  Video super-resolution (SR) aims at generating a sequence of high-resolution
(HR) frames with plausible and temporally consistent details from their
low-resolution (LR) counterparts. The key challenge for video SR lies in the
effective exploitation of temporal dependency between consecutive frames.
Existing deep learning based methods commonly estimate optical flows between LR
frames to provide temporal dependency. However, the resolution conflict between
LR optical flows and HR outputs hinders the recovery of fine details. In this
paper, we propose an end-to-end video SR network to super-resolve both optical
flows and images. Optical flow SR from LR frames provides accurate temporal
dependency and ultimately improves video SR performance. Specifically, we first
propose an optical flow reconstruction network (OFRnet) to infer HR optical
flows in a coarse-to-fine manner. Then, motion compensation is performed using
HR optical flows to encode temporal dependency. Finally, compensated LR inputs
are fed to a super-resolution network (SRnet) to generate SR results. Extensive
experiments have been conducted to demonstrate the effectiveness of HR optical
flows for SR performance improvement. Comparative results on the Vid4 and
DAVIS-10 datasets show that our network achieves the state-of-the-art
performance.
