  We propose Stereo Direct Sparse Odometry (Stereo DSO) as a novel method for
highly accurate real-time visual odometry estimation of large-scale
environments from stereo cameras. It jointly optimizes for all the model
parameters within the active window, including the intrinsic/extrinsic camera
parameters of all keyframes and the depth values of all selected pixels. In
particular, we propose a novel approach to integrate constraints from static
stereo into the bundle adjustment pipeline of temporal multi-view stereo.
Real-time optimization is realized by sampling pixels uniformly from image
regions with sufficient intensity gradient. Fixed-baseline stereo resolves
scale drift. It also reduces the sensitivities to large optical flow and to
rolling shutter effect which are known shortcomings of direct image alignment
methods. Quantitative evaluation demonstrates that the proposed Stereo DSO
outperforms existing state-of-the-art visual odometry methods both in terms of
tracking accuracy and robustness. Moreover, our method delivers a more precise
metric 3D reconstruction than previous dense/semi-dense direct approaches while
providing a higher reconstruction density than feature-based methods.
