  Constructing effective image priors is critical to solving ill-posed inverse
problems in image processing and imaging. Recent works proposed to exploit
image non-local similarity for inverse problems by grouping similar patches and
demonstrated state-of-the-art results in many applications. However, compared
to classic methods based on filtering or sparsity, most of the non-local
algorithms are time-consuming, mainly due to the highly inefficient and
redundant block matching step, where the distance between each pair of
overlapping patches needs to be computed. In this work, we propose a novel
Self-Convolution operator to exploit image non-local similarity in a
self-supervised way. The proposed Self-Convolution can generalize the
commonly-used block matching step and produce equivalent results with much
cheaper computation. Furthermore, by applying Self-Convolution, we propose an
effective multi-modality image restoration scheme, which is much more efficient
than conventional block matching for non-local modeling. Experimental results
demonstrate that (1) Self-Convolution can significantly speed up most of the
popular non-local image restoration algorithms, with two-fold to nine-fold
faster block matching, and (2) the proposed multi-modality image restoration
scheme achieves superior denoising results in both efficiency and effectiveness
on RGB-NIR images. The code is publicly available at
\href{https://github.com/GuoLanqing/Self-Convolution}.
