  As an important and challenging problem in computer vision, learning based
optical flow estimation aims to discover the intrinsic correspondence structure
between two adjacent video frames through statistical learning. Therefore, a
key issue to solve in this area is how to effectively model the multi-scale
correspondence structure properties in an adaptive end-to-end learning fashion.
Motivated by this observation, we propose an end-to-end multi-scale
correspondence structure learning (MSCSL) approach for optical flow estimation.
In principle, the proposed MSCSL approach is capable of effectively capturing
the multi-scale inter-image-correlation correspondence structures within a
multi-level feature space from deep learning. Moreover, the proposed MSCSL
approach builds a spatial Conv-GRU neural network model to adaptively model the
intrinsic dependency relationships among these multi-scale correspondence
structures. Finally, the above procedures for correspondence structure learning
and multi-scale dependency modeling are implemented in a unified end-to-end
deep learning framework. Experimental results on several benchmark datasets
demonstrate the effectiveness of the proposed approach.
