  Existing video super-resolution (SR) algorithms usually assume that the blur
kernels in the degradation process are known and do not model the blur kernels
in the restoration. However, this assumption does not hold for video SR and
usually leads to over-smoothed super-resolved images. In this paper, we propose
a deep convolutional neural network (CNN) model to solve video SR by a blur
kernel modeling approach. The proposed deep CNN model consists of motion blur
estimation, motion estimation, and latent image restoration modules. The motion
blur estimation module is used to provide reliable blur kernels. With the
estimated blur kernel, we develop an image deconvolution method based on the
image formation model of video SR to generate intermediate latent images so
that some sharp image contents can be restored well. However, the generated
intermediate latent images may contain artifacts. To generate high-quality
images, we use the motion estimation module to explore the information from
adjacent frames, where the motion estimation can constrain the deep CNN model
for better image restoration. We show that the proposed algorithm is able to
generate clearer images with finer structural details. Extensive experimental
results show that the proposed algorithm performs favorably against
state-of-the-art methods.
