  Image guidance is an effective strategy for depth super-resolution.
Generally, most existing methods employ hand-crafted operators to decompose the
high-frequency (HF) and low-frequency (LF) ingredients from low-resolution
depth maps and guide the HF ingredients by directly concatenating them with
image features. However, the hand-designed operators usually cause inferior HF
maps (e.g., distorted or structurally missing) due to the diverse appearance of
complex depth maps. Moreover, the direct concatenation often results in weak
guidance because not all image features have a positive effect on the HF maps.
In this paper, we develop a recurrent structure attention guided (RSAG)
framework, consisting of two important parts. First, we introduce a deep
contrastive network with multi-scale filters for adaptive frequency-domain
separation, which adopts contrastive networks from large filters to small ones
to calculate the pixel contrasts for adaptive high-quality HF predictions.
Second, instead of the coarse concatenation guidance, we propose a recurrent
structure attention block, which iteratively utilizes the latest depth
estimation and the image features to jointly select clear patterns and
boundaries, aiming at providing refined guidance for accurate depth recovery.
In addition, we fuse the features of HF maps to enhance the edge structures in
the decomposed LF maps. Extensive experiments show that our approach obtains
superior performance compared with state-of-the-art depth super-resolution
methods.
