  In this paper, we propose a novel end-to-end deep neural network model for
omnidirectional depth estimation from a wide-baseline multi-view stereo setup.
The images captured with ultra wide field-of-view (FOV) cameras on an
omnidirectional rig are processed by the feature extraction module, and then
the deep feature maps are warped onto the concentric spheres swept through all
candidate depths using the calibrated camera parameters. The 3D encoder-decoder
block takes the aligned feature volume to produce the omnidirectional depth
estimate with regularization on uncertain regions utilizing the global context
information. In addition, we present large-scale synthetic datasets for
training and testing omnidirectional multi-view stereo algorithms. Our datasets
consist of 11K ground-truth depth maps and 45K fisheye images in four
orthogonal directions with various objects and environments. Experimental
results show that the proposed method generates excellent results in both
synthetic and real-world environments, and it outperforms the prior art and the
omnidirectional versions of the state-of-the-art conventional stereo
algorithms.
