  We present a real-time dense geometric mapping algorithm for large-scale
environments. Unlike existing methods which use pinhole cameras, our
implementation is based on fisheye cameras which have larger field of view and
benefit some other tasks including Visual-Inertial Odometry, localization and
object detection around vehicles. Our algorithm runs on in-vehicle PCs at 15 Hz
approximately, enabling vision-only 3D scene perception for self-driving
vehicles. For each synchronized set of images captured by multiple cameras, we
first compute a depth map for a reference camera using plane-sweeping stereo.
To maintain both accuracy and efficiency, while accounting for the fact that
fisheye images have a rather low resolution, we recover the depths using
multiple image resolutions. We adopt the fast object detection framework YOLOv3
to remove potentially dynamic objects. At the end of the pipeline, we fuse the
fisheye depth images into the truncated signed distance function (TSDF) volume
to obtain a 3D map. We evaluate our method on large-scale urban datasets, and
results show that our method works well even in complex environments.
