  Guided sparse depth upsampling aims to upsample an irregularly sampled sparse
depth map when an aligned high-resolution color image is given as guidance.
Many neural networks have been designed for this task. However, they often
ignore the structural difference between the depth and the color image,
resulting in obvious artifacts such as texture copy and depth blur at the
upsampling depth. Inspired by the normalized convolution operation, we propose
a guided convolutional layer to recover dense depth from sparse and irregular
depth image with an depth edge image as guidance. Our novel guided network can
prevent the depth value from crossing the depth edge to facilitate upsampling.
We further design a convolution network based on proposed convolutional layer
to combine the advantages of different algorithms and achieve better
performance. We conduct comprehensive experiments to verify our method on
real-world indoor and synthetic outdoor datasets. Our method produces strong
results. It outperforms state-of-the-art methods on the Virtual KITTI dataset
and the Middlebury dataset. It also presents strong generalization capability
under different 3D point densities, various lighting and weather conditions.
