  The ability of accurate depth prediction by a convolutional neural network
(CNN) is a major challenge for its wide use in practical visual simultaneous
localization and mapping (SLAM) applications, such as enhanced camera tracking
and dense mapping. This paper is set out to answer the following question: Can
we tune a depth prediction CNN with the help of a visual SLAM algorithm even if
the CNN is not trained for the current operating environment in order to
benefit the SLAM performance? To this end, we propose a novel online adaptation
framework consisting of two complementary processes: a SLAM algorithm that is
used to generate keyframes to fine-tune the depth prediction and another
algorithm that uses the online adapted depth to improve map quality. Once the
potential noisy map points are removed, we perform global photometric bundle
adjustment (BA) to improve the overall SLAM performance. Experimental results
on both benchmark datasets and a real robot in our own experimental
environments show that our proposed method improves the overall SLAM accuracy.
While regularization has been shown to be effective in multi-task
classification problems, we present experimental results and an ablation study
to show the effectiveness of regularization in preventing catastrophic
forgetting in the online adaptation of depth prediction, a single-task
regression problem. In addition, we compare our online adaptation framework
against the state-of-the-art pre-trained depth prediction CNNs to show that our
online adapted depth prediction CNN outperforms the depth prediction CNNs that
have been trained on a large collection of datasets.
