  The performance of video frame interpolation is inherently correlated with
the ability to handle motion in the input scene. Even though previous works
recognize the utility of asynchronous event information for this task, they
ignore the fact that motion may or may not result in blur in the input video to
be interpolated, depending on the length of the exposure time of the frames and
the speed of the motion, and assume either that the input video is sharp,
restricting themselves to frame interpolation, or that it is blurry, including
an explicit, separate deblurring stage before interpolation in their pipeline.
We instead propose a general method for event-based frame interpolation that
performs deblurring ad-hoc and thus works both on sharp and blurry input
videos. Our model consists in a bidirectional recurrent network that naturally
incorporates the temporal dimension of interpolation and fuses information from
the input frames and the events adaptively based on their temporal proximity.
In addition, we introduce a novel real-world high-resolution dataset with
events and color videos named HighREV, which provides a challenging evaluation
setting for the examined task. Extensive experiments on the standard GoPro
benchmark and on our dataset show that our network consistently outperforms
previous state-of-the-art methods on frame interpolation, single image
deblurring and the joint task of interpolation and deblurring. Our code and
dataset will be made publicly available.
