  In this study, a deep-learning-based multi-stage network architecture called
Multi-Stage Depth Prediction Network (MSDPN) is proposed to predict a dense
depth map using a 2D LiDAR and a monocular camera. Our proposed network
consists of a multi-stage encoder-decoder architecture and Cross Stage Feature
Aggregation (CSFA). The proposed multi-stage encoder-decoder architecture
alleviates the partial observation problem caused by the characteristics of a
2D LiDAR, and CSFA prevents the multi-stage network from diluting the features
and allows the network to learn the inter-spatial relationship between features
better. Previous works use sub-sampled data from the ground truth as an input
rather than actual 2D LiDAR data. In contrast, our approach trains the model
and conducts experiments with a physically-collected 2D LiDAR dataset. To this
end, we acquired our own dataset called KAIST RGBD-scan dataset and validated
the effectiveness and the robustness of MSDPN under realistic conditions. As
verified experimentally, our network yields promising performance against
state-of-the-art methods. Additionally, we analyzed the performance of
different input methods and confirmed that the reference depth map is robust in
untrained scenarios.
