  Recently, Convolutional Neural Networks (CNN) based image super-resolution
(SR) have shown significant success in the literature. However, these methods
are implemented as single-path stream to enrich feature maps from the input for
the final prediction, which fail to fully incorporate former low-level features
into later high-level features. In this paper, to tackle this problem, we
propose a deep interleaved network (DIN) to learn how information at different
states should be combined for image SR where shallow information guides deep
representative features prediction. Our DIN follows a multi-branch pattern
allowing multiple interconnected branches to interleave and fuse at different
states. Besides, the asymmetric co-attention (AsyCA) is proposed and attacked
to the interleaved nodes to adaptively emphasize informative features from
different states and improve the discriminative ability of networks. Extensive
experiments demonstrate the superiority of our proposed DIN in comparison with
the state-of-the-art SR methods.
