  Depth completion aims to recover a dense depth map from a sparse depth map
with the corresponding color image as input. Recent approaches mainly formulate
depth completion as a one-stage end-to-end learning task, which outputs dense
depth maps directly. However, the feature extraction and supervision in
one-stage frameworks are insufficient, limiting the performance of these
approaches. To address this problem, we propose a novel end-to-end residual
learning framework, which formulates the depth completion as a two-stage
learning task, i.e., a sparse-to-coarse stage and a coarse-to-fine stage.
First, a coarse dense depth map is obtained by a simple CNN framework. Then, a
refined depth map is further obtained using a residual learning strategy in the
coarse-to-fine stage with a coarse depth map and color image as input.
Specially, in the coarse-to-fine stage, a channel shuffle extraction operation
is utilized to extract more representative features from the color image and
coarse depth map, and an energy based fusion operation is exploited to
effectively fuse these features obtained by channel shuffle operation, thus
leading to more accurate and refined depth maps. We achieve SoTA performance in
RMSE on KITTI benchmark. Extensive experiments on other datasets future
demonstrate the superiority of our approach over current state-of-the-art depth
completion approaches.
