  In this paper, we present a multi-camera visual odometry (VO) system for an
autonomous vehicle. Our system mainly consists of a virtual LiDAR and a pose
tracker. We use a perspective transformation method to synthesize a
surround-view image from undistorted fisheye camera images. With a semantic
segmentation model, the free space can be extracted. The scans of the virtual
LiDAR are generated by discretizing the contours of the free space. As for the
pose tracker, we propose a visual odometry system fusing both the feature
matching and the virtual LiDAR scan matching results. Only those feature points
located in the free space area are utilized to ensure the 2D-2D matching for
pose estimation. Furthermore, bundle adjustment (BA) is performed to minimize
the feature points reprojection error and scan matching error. We apply our
system to an autonomous vehicle equipped with four fisheye cameras. The testing
scenarios include an outdoor parking lot as well as an indoor garage.
Experimental results demonstrate that our system achieves a more robust and
accurate performance comparing with a fisheye camera based monocular visual
odometry system.
