  Motion estimation (ME) and motion compensation (MC) have been widely used for
classical video frame interpolation systems over the past decades. Recently, a
number of data-driven frame interpolation methods based on convolutional neural
networks have been proposed. However, existing learning based methods typically
estimate either flow or compensation kernels, thereby limiting performance on
both computational efficiency and interpolation accuracy. In this work, we
propose a motion estimation and compensation driven neural network for video
frame interpolation. A novel adaptive warping layer is developed to integrate
both optical flow and interpolation kernels to synthesize target frame pixels.
This layer is fully differentiable such that both the flow and kernel
estimation networks can be optimized jointly. The proposed model benefits from
the advantages of motion estimation and compensation methods without using
hand-crafted features. Compared to existing methods, our approach is
computationally efficient and able to generate more visually appealing results.
Furthermore, the proposed MEMC-Net can be seamlessly adapted to several video
enhancement tasks, e.g., super-resolution, denoising, and deblocking. Extensive
quantitative and qualitative evaluations demonstrate that the proposed method
performs favorably against the state-of-the-art video frame interpolation and
enhancement algorithms on a wide range of datasets.
