  Depth map records distance between the viewpoint and objects in the scene,
which plays a critical role in many real-world applications. However, depth map
captured by consumer-grade RGB-D cameras suffers from low spatial resolution.
Guided depth map super-resolution (DSR) is a popular approach to address this
problem, which attempts to restore a high-resolution (HR) depth map from the
input low-resolution (LR) depth and its coupled HR RGB image that serves as the
guidance.
  The most challenging problems for guided DSR are how to correctly select
consistent structures and propagate them, and properly handle inconsistent
ones. In this paper, we propose a novel attention-based hierarchical
multi-modal fusion (AHMF) network for guided DSR. Specifically, to effectively
extract and combine relevant information from LR depth and HR guidance, we
propose a multi-modal attention based fusion (MMAF) strategy for hierarchical
convolutional layers, including a feature enhance block to select valuable
features and a feature recalibration block to unify the similarity metrics of
modalities with different appearance characteristics. Furthermore, we propose a
bi-directional hierarchical feature collaboration (BHFC) module to fully
leverage low-level spatial information and high-level structure information
among multi-scale features. Experimental results show that our approach
outperforms state-of-the-art methods in terms of reconstruction accuracy,
running speed and memory efficiency.
