  We propose DSP-SLAM, an object-oriented SLAM system that builds a rich and
accurate joint map of dense 3D models for foreground objects, and sparse
landmark points to represent the background. DSP-SLAM takes as input the 3D
point cloud reconstructed by a feature-based SLAM system and equips it with the
ability to enhance its sparse map with dense reconstructions of detected
objects. Objects are detected via semantic instance segmentation, and their
shape and pose is estimated using category-specific deep shape embeddings as
priors, via a novel second order optimization. Our object-aware bundle
adjustment builds a pose-graph to jointly optimize camera poses, object
locations and feature points. DSP-SLAM can operate at 10 frames per second on 3
different input modalities: monocular, stereo, or stereo+LiDAR. We demonstrate
DSP-SLAM operating at almost frame rate on monocular-RGB sequences from the
Friburg and Redwood-OS datasets, and on stereo+LiDAR sequences on the KITTI
odometry dataset showing that it achieves high-quality full object
reconstructions, even from partial observations, while maintaining a consistent
global map. Our evaluation shows improvements in object pose and shape
reconstruction with respect to recent deep prior-based reconstruction methods
and reductions in camera tracking drift on the KITTI dataset.
