  The employment of convolutional neural networks has achieved unprecedented
performance in the task of image restoration for a variety of degradation
factors. However, high-performance networks have been specifically designed for
a single degradation factor. In this paper, we tackle a harder problem,
restoring a clean image from its degraded version with an unknown degradation
factor, subject to the condition that it is one of the known factors. Toward
this end, we design a network having multiple pairs of input and output
branches and use it in a recurrent fashion such that a different branch pair is
used at each of the recurrent paths. We reinforce the shared part of the
network with improved components so that it can handle different degradation
factors. We also propose a two-step training method for the network, which
consists of multi-task learning and finetuning. The experimental results show
that the proposed network yields at least comparable or sometimes even better
performance on four degradation factors as compared with the best dedicated
network for each of the four. We also test it on a further harder task where
the input image contains multiple degradation factors that are mixed with
unknown mixture ratios, showing that it achieves better performance than the
previous state-of-the-art method designed for the task.
