  Coarse-to-fine strategies have been extensively used for the architecture
design of single image deblurring networks. Conventional methods typically
stack sub-networks with multi-scale input images and gradually improve
sharpness of images from the bottom sub-network to the top sub-network,
yielding inevitably high computational costs. Toward a fast and accurate
deblurring network design, we revisit the coarse-to-fine strategy and present a
multi-input multi-output U-net (MIMO-UNet). The MIMO-UNet has three distinct
features. First, the single encoder of the MIMO-UNet takes multi-scale input
images to ease the difficulty of training. Second, the single decoder of the
MIMO-UNet outputs multiple deblurred images with different scales to mimic
multi-cascaded U-nets using a single U-shaped network. Last, asymmetric feature
fusion is introduced to merge multi-scale features in an efficient manner.
Extensive experiments on the GoPro and RealBlur datasets demonstrate that the
proposed network outperforms the state-of-the-art methods in terms of both
accuracy and computational complexity. Source code is available for research
purposes at https://github.com/chosj95/MIMO-UNet.
