  Video deblurring is a challenging task due to the spatially variant blur
caused by camera shake, object motions, and depth variations, etc. Existing
methods usually estimate optical flow in the blurry video to align consecutive
frames or approximate blur kernels. However, they tend to generate artifacts or
cannot effectively remove blur when the estimated optical flow is not accurate.
To overcome the limitation of separate optical flow estimation, we propose a
Spatio-Temporal Filter Adaptive Network (STFAN) for the alignment and
deblurring in a unified framework. The proposed STFAN takes both blurry and
restored images of the previous frame as well as blurry image of the current
frame as input, and dynamically generates the spatially adaptive filters for
the alignment and deblurring. We then propose the new Filter Adaptive
Convolutional (FAC) layer to align the deblurred features of the previous frame
with the current frame and remove the spatially variant blur from the features
of the current frame. Finally, we develop a reconstruction network which takes
the fusion of two transformed features to restore the clear frames. Both
quantitative and qualitative evaluation results on the benchmark datasets and
real-world videos demonstrate that the proposed algorithm performs favorably
against state-of-the-art methods in terms of accuracy, speed as well as model
size.
