  In contrast to traditional image restoration methods, all-in-one image
restoration techniques are gaining increased attention for their ability to
restore images affected by diverse and unknown corruption types and levels.
However, contemporary all-in-one image restoration methods omit task-wise
difficulties and employ the same networks to reconstruct images afflicted by
diverse degradations. This practice leads to an underestimation of the task
correlations and suboptimal allocation of computational resources. To elucidate
task-wise complexities, we introduce a novel concept positing that intricate
image degradation can be represented in terms of elementary degradation.
Building upon this foundation, we propose an innovative approach, termed the
Unified-Width Adaptive Dynamic Network (U-WADN), consisting of two pivotal
components: a Width Adaptive Backbone (WAB) and a Width Selector (WS). The WAB
incorporates several nested sub-networks with varying widths, which facilitates
the selection of the most apt computations tailored to each task, thereby
striking a balance between accuracy and computational efficiency during
runtime. For different inputs, the WS automatically selects the most
appropriate sub-network width, taking into account both task-specific and
sample-specific complexities. Extensive experiments across a variety of image
restoration tasks demonstrate that the proposed U-WADN achieves better
performance while simultaneously reducing up to 32.3\% of FLOPs and providing
approximately 15.7\% real-time acceleration. The code has been made available
at \url{https://github.com/xuyimin0926/U-WADN}.
