  This paper presents a new deep-learning based method to simultaneously
calibrate the intrinsic parameters of fisheye lens and rectify the distorted
images. Assuming that the distorted lines generated by fisheye projection
should be straight after rectification, we propose a novel deep neural network
to impose explicit geometry constraints onto processes of the fisheye lens
calibration and the distorted image rectification. In addition, considering the
nonlinearity of distortion distribution in fisheye images, the proposed network
fully exploits multi-scale perception to equalize the rectification effects on
the whole image. To train and evaluate the proposed model, we also create a new
largescale dataset labeled with corresponding distortion parameters and
well-annotated distorted lines. Compared with the state-of-the-art methods, our
model achieves the best published rectification quality and the most accurate
estimation of distortion parameters on a large set of synthetic and real
fisheye images.
