  Optical flow estimation is an important yet challenging problem in the field
of video analytics. The features of different semantics levels/layers of a
convolutional neural network can provide information of different granularity.
To exploit such flexible and comprehensive information, we propose a
semi-supervised Feature Pyramidal Correlation and Residual Reconstruction
Network (FPCR-Net) for optical flow estimation from frame pairs. It consists of
two main modules: pyramid correlation mapping and residual reconstruction. The
pyramid correlation mapping module takes advantage of the multi-scale
correlations of global/local patches by aggregating features of different
scales to form a multi-level cost volume. The residual reconstruction module
aims to reconstruct the sub-band high-frequency residuals of finer optical flow
in each stage. Based on the pyramid correlation mapping, we further propose a
correlation-warping-normalization (CWN) module to efficiently exploit the
correlation dependency. Experiment results show that the proposed scheme
achieves the state-of-the-art performance, with improvement by 0.80, 1.15 and
0.10 in terms of average end-point error (AEE) against competing baseline
methods - FlowNet2, LiteFlowNet and PWC-Net on the Final pass of Sintel
dataset, respectively.
