  Regularization-based image restoration has remained an active research topic
in computer vision and image processing. It often leverages a guidance signal
captured in different fields as an additional cue. In this work, we present a
general framework for image restoration, called deeply aggregated alternating
minimization (DeepAM). We propose to train deep neural network to advance two
of the steps in the conventional AM algorithm: proximal mapping and ?-
continuation. Both steps are learned from a large dataset in an end-to-end
manner. The proposed framework enables the convolutional neural networks (CNNs)
to operate as a prior or regularizer in the AM algorithm. We show that our
learned regularizer via deep aggregation outperforms the recent data-driven
approaches as well as the nonlocalbased methods. The flexibility and
effectiveness of our framework are demonstrated in several image restoration
tasks, including single image denoising, RGB-NIR restoration, and depth
super-resolution.
