  Recently, convolutional neural network (CNN) has demonstrated significant
success for image restoration (IR) tasks (e.g., image super-resolution, image
deblurring, rain streak removal, and dehazing). However, existing CNN based
models are commonly implemented as a single-path stream to enrich feature
representations from low-quality (LQ) input space for final predictions, which
fail to fully incorporate preceding low-level contexts into later high-level
features within networks, thereby producing inferior results. In this paper, we
present a deep interleaved network (DIN) that learns how information at
different states should be combined for high-quality (HQ) images
reconstruction. The proposed DIN follows a multi-path and multi-branch pattern
allowing multiple interconnected branches to interleave and fuse at different
states. In this way, the shallow information can guide deep representative
features prediction to enhance the feature expression ability. Furthermore, we
propose asymmetric co-attention (AsyCA) which is attached at each interleaved
node to model the feature dependencies. Such AsyCA can not only adaptively
emphasize the informative features from different states, but also improves the
discriminative ability of networks. Our presented DIN can be trained end-to-end
and applied to various IR tasks. Comprehensive evaluations on public benchmarks
and real-world datasets demonstrate that the proposed DIN perform favorably
against the state-of-the-art methods quantitatively and qualitatively.
