  Image restoration is a long-standing low-level vision problem, e.g.,
deblurring and deraining. In the process of image restoration, it is necessary
to consider not only the spatial details and contextual information of
restoration to ensure the quality, but also the system complexity. Although
many methods have been able to guarantee the quality of image restoration, the
system complexity of the state-of-the-art (SOTA) methods is increasing as well.
Motivated by this, we present a mixed hierarchy network that can balance these
competing goals. Our main proposal is a mixed hierarchy architecture, that
progressively recovers contextual information and spatial details from degraded
images while we design intra-blocks to reduce system complexity. Specifically,
our model first learns the contextual information using encoder-decoder
architectures, and then combines them with high-resolution branches that
preserve spatial detail. In order to reduce the system complexity of this
architecture for convenient analysis and comparison, we replace or remove the
nonlinear activation function with multiplication and use a simple network
structure. In addition, we replace spatial convolution with global
self-attention for the middle block of encoder-decoder. The resulting tightly
interlinked hierarchy architecture, named as MHNet, delivers strong performance
gains on several image restoration tasks, including image deraining, and
deblurring.
