  Recent studies show that, without any prior model, the unsupervised
restoration learning problem can be optimally formulated as an optimal
transport (OT) problem, which has shown promising performance on denoising
tasks to approach the performance of supervised methods. However, it still
significantly lags behind state-of-the-art supervised methods on complex
restoration tasks such as super-resolution, deraining, and dehazing. In this
paper, we exploit the sparsity of degradation in the OT framework to
significantly boost its performance on these tasks. First, we disclose an
observation that the degradation in these tasks is quite sparse in the
frequency domain, and then propose a sparsity-aware optimal transport (SOT)
criterion for unsupervised restoration learning. Further, we provide an
analytic example to illustrate that exploiting the sparsity helps to reduce the
ambiguity in finding an inverse map for restoration. Experiments on real-world
super-resolution, deraining, and dehazing demonstrate that SOT can improve the
PSNR of OT by about 2.6 dB, 2.7 dB and 1.3 dB, respectively, while achieving
the best perception scores among the compared supervised and unsupervised
methods. Particularly, on the three tasks, SOT significantly outperforms
existing unsupervised methods and approaches the performance of
state-of-the-art supervised methods.
