  The performance of single image super-resolution has achieved significant
improvement by utilizing deep convolutional neural networks (CNNs). The
features in deep CNN contain different types of information which make
different contributions to image reconstruction. However, most CNN-based models
lack discriminative ability for different types of information and deal with
them equally, which results in the representational capacity of the models
being limited. On the other hand, as the depth of neural networks grows, the
long-term information coming from preceding layers is easy to be weaken or lost
in late layers, which is adverse to super-resolving image. To capture more
informative features and maintain long-term information for image
super-resolution, we propose a channel-wise and spatial feature modulation
(CSFM) network in which a sequence of feature-modulation memory (FMM) modules
is cascaded with a densely connected structure to transform low-resolution
features to high informative features. In each FMM module, we construct a set
of channel-wise and spatial attention residual (CSAR) blocks and stack them in
a chain structure to dynamically modulate multi-level features in a
global-and-local manner. This feature modulation strategy enables the high
contribution information to be enhanced and the redundant information to be
suppressed. Meanwhile, for long-term information persistence, a gated fusion
(GF) node is attached at the end of the FMM module to adaptively fuse
hierarchical features and distill more effective information via the dense skip
connections and the gating mechanism. Extensive quantitative and qualitative
evaluations on benchmark datasets illustrate the superiority of our proposed
method over the state-of-the-art methods.
