  We propose D3VO as a novel framework for monocular visual odometry that
exploits deep networks on three levels -- deep depth, pose and uncertainty
estimation. We first propose a novel self-supervised monocular depth estimation
network trained on stereo videos without any external supervision. In
particular, it aligns the training image pairs into similar lighting condition
with predictive brightness transformation parameters. Besides, we model the
photometric uncertainties of pixels on the input images, which improves the
depth estimation accuracy and provides a learned weighting function for the
photometric residuals in direct (feature-less) visual odometry. Evaluation
results show that the proposed network outperforms state-of-the-art
self-supervised depth estimation networks. D3VO tightly incorporates the
predicted depth, pose and uncertainty into a direct visual odometry method to
boost both the front-end tracking as well as the back-end non-linear
optimization. We evaluate D3VO in terms of monocular visual odometry on both
the KITTI odometry benchmark and the EuRoC MAV dataset.The results show that
D3VO outperforms state-of-the-art traditional monocular VO methods by a large
margin. It also achieves comparable results to state-of-the-art stereo/LiDAR
odometry on KITTI and to the state-of-the-art visual-inertial odometry on EuRoC
MAV, while using only a single camera.
