  Video Frame Interpolation synthesizes non-existent images between adjacent
frames, with the aim of providing a smooth and consistent visual experience.
Two approaches for solving this challenging task are optical flow based and
kernel-based methods. In existing works, optical flow based methods can provide
accurate point-to-point motion description, however, they lack constraints on
object structure. On the contrary, kernel-based methods focus on structural
alignment, which relies on semantic and apparent features, but tends to blur
results. Based on these observations, we propose a structure-motion based
iterative fusion method. The framework is an end-to-end learnable structure
with two stages. First, interpolated frames are synthesized by structure-based
and motion-based learning branches respectively, then, an iterative refinement
module is established via spatial and temporal feature integration. Inspired by
the observation that audiences have different visual preferences on foreground
and background objects, we for the first time propose to use saliency masks in
the evaluation processes of the task of video frame interpolation. Experimental
results on three typical benchmarks show that the proposed method achieves
superior performance on all evaluation metrics over the state-of-the-art
methods, even when our models are trained with only one-tenth of the data other
methods use.
