  Most existing non-blind restoration methods are based on the assumption that
a precise degradation model is known. As the degradation process can only be
partially known or inaccurately modeled, images may not be well restored. Rain
streak removal and image deconvolution with inaccurate blur kernels are two
representative examples of such tasks. For rain streak removal, although an
input image can be decomposed into a scene layer and a rain streak layer, there
exists no explicit formulation for modeling rain streaks and the composition
with scene layer. For blind deconvolution, as estimation error of blur kernel
is usually introduced, the subsequent non-blind deconvolution process does not
restore the latent image well. In this paper, we propose a principled algorithm
within the maximum a posterior framework to tackle image restoration with a
partially known or inaccurate degradation model. Specifically, the residual
caused by a partially known or inaccurate degradation model is spatially
dependent and complexly distributed. With a training set of degraded and
ground-truth image pairs, we parameterize and learn the fidelity term for a
degradation model in a task-driven manner. Furthermore, the regularization term
can also be learned along with the fidelity term, thereby forming a
simultaneous fidelity and regularization learning model. Extensive experimental
results demonstrate the effectiveness of the proposed model for image
deconvolution with inaccurate blur kernels, deconvolution with multiple
degradations and rain streak removal.
