  Deep learning based single image super-resolution methods use a large number
of training datasets and have recently achieved great quality progress both
quantitatively and qualitatively. Most deep networks focus on nonlinear mapping
from low-resolution inputs to high-resolution outputs via residual learning
without exploring the feature abstraction and analysis. We propose a
Hierarchical Back Projection Network (HBPN), that cascades multiple HourGlass
(HG) modules to bottom-up and top-down process features across all scales to
capture various spatial correlations and then consolidates the best
representation for reconstruction. We adopt the back projection blocks in our
proposed network to provide the error correlated up and down-sampling process
to replace simple deconvolution and pooling process for better estimation. A
new Softmax based Weighted Reconstruction (WR) process is used to combine the
outputs of HG modules to further improve super-resolution. Experimental results
on various datasets (including the validation dataset, NTIRE2019, of the Real
Image Super-resolution Challenge) show that our proposed approach can achieve
and improve the performance of the state-of-the-art methods for different
scaling factors.
