  Computer vision researchers have been expecting that neural networks have
spatial transformation ability to eliminate the interference caused by
geometric distortion for a long time. Emergence of spatial transformer network
makes dream come true. Spatial transformer network and its variants can handle
global displacement well, but lack the ability to deal with local spatial
variance. Hence how to achieve a better manner of deformation in the neural
network has become a pressing matter of the moment. To address this issue, we
analyze the advantages and disadvantages of approximation theory and optical
flow theory, then we combine them to propose a novel way to achieve image
deformation and implement it with a hierarchical convolutional neural network.
This new approach solves for a linear deformation along with an optical flow
field to model image deformation. In the experiments of cluttered MNIST
handwritten digits classification and image plane alignment, our method
outperforms baseline methods by a large margin.
