  Real-world image super-resolution (SR) tasks often do not have paired
datasets, which limits the application of supervised techniques. As a result,
the tasks are usually approached by unpaired techniques based on Generative
Adversarial Networks (GANs), which yield complex training losses with several
regularization terms, e.g., content or identity losses. We theoretically
investigate optimization problems which arise in such models and find two
surprizing observations. First, the learned SR map is always an optimal
transport (OT) map. Second, we theoretically prove and empirically show that
the learned map is biased, i.e., it does not actually transform the
distribution of low-resolution images to high-resolution ones. Inspired by
these findings, we propose an algorithm for unpaired SR which learns an
unbiased OT map for the perceptual transport cost. Unlike the existing
GAN-based alternatives, our algorithm has a simple optimization objective
reducing the need for complex hyperparameter selection and an application of
additional regularizations. At the same time, it provides a nearly
state-of-the-art performance on the large-scale unpaired AIM19 dataset.
