  Despite the remarkable progresses made in deep-learning based depth map
super-resolution (DSR), how to tackle real-world degradation in low-resolution
(LR) depth maps remains a major challenge. Existing DSR model is generally
trained and tested on synthetic dataset, which is very different from what
would get from a real depth sensor. In this paper, we argue that DSR models
trained under this setting are restrictive and not effective in dealing with
real-world DSR tasks. We make two contributions in tackling real-world
degradation of different depth sensors. First, we propose to classify the
generation of LR depth maps into two types: non-linear downsampling with noise
and interval downsampling, for which DSR models are learned correspondingly.
Second, we propose a new framework for real-world DSR, which consists of four
modules : 1) An iterative residual learning module with deep supervision to
learn effective high-frequency components of depth maps in a coarse-to-fine
manner; 2) A channel attention strategy to enhance channels with abundant
high-frequency components; 3) A multi-stage fusion module to effectively
re-exploit the results in the coarse-to-fine process; and 4) A depth refinement
module to improve the depth map by TGV regularization and input loss. Extensive
experiments on benchmarking datasets demonstrate the superiority of our method
over current state-of-the-art DSR methods.
