  Deep learning based localization and mapping has recently attracted
significant attention. Instead of creating hand-designed algorithms through
exploitation of physical models or geometric theories, deep learning based
solutions provide an alternative to solve the problem in a data-driven way.
Benefiting from ever-increasing volumes of data and computational power, these
methods are fast evolving into a new area that offers accurate and robust
systems to track motion and estimate scenes and their structure for real-world
applications. In this work, we provide a comprehensive survey, and propose a
new taxonomy for localization and mapping using deep learning. We also discuss
the limitations of current models, and indicate possible future directions. A
wide range of topics are covered, from learning odometry estimation, mapping,
to global localization and simultaneous localization and mapping (SLAM). We
revisit the problem of perceiving self-motion and scene understanding with
on-board sensors, and show how to solve it by integrating these modules into a
prospective spatial machine intelligence system (SMIS). It is our hope that
this work can connect emerging works from robotics, computer vision and machine
learning communities, and serve as a guide for future researchers to apply deep
learning to tackle localization and mapping problems.
