  We propose a novel dense mapping framework for sparse visual SLAM systems
which leverages a compact scene representation. State-of-the-art sparse visual
SLAM systems provide accurate and reliable estimates of the camera trajectory
and locations of landmarks. While these sparse maps are useful for
localization, they cannot be used for other tasks such as obstacle avoidance or
scene understanding. In this paper we propose a dense mapping framework to
complement sparse visual SLAM systems which takes as input the camera poses,
keyframes and sparse points produced by the SLAM system and predicts a dense
depth image for every keyframe. We build on CodeSLAM and use a variational
autoencoder (VAE) which is conditioned on intensity, sparse depth and
reprojection error images from sparse SLAM to predict an uncertainty-aware
dense depth map. The use of a VAE then enables us to refine the dense depth
images through multi-view optimization which improves the consistency of
overlapping frames. Our mapper runs in a separate thread in parallel to the
SLAM system in a loosely coupled manner. This flexible design allows for
integration with arbitrary metric sparse SLAM systems without delaying the main
SLAM process. Our dense mapper can be used not only for local mapping but also
globally consistent dense 3D reconstruction through TSDF fusion. We demonstrate
our system running with ORB-SLAM3 and show accurate dense depth estimation
which could enable applications such as robotics and augmented reality.
