  Most learning-based super-resolution (SR) methods aim to recover
high-resolution (HR) image from a given low-resolution (LR) image via learning
on LR-HR image pairs. The SR methods learned on synthetic data do not perform
well in real-world, due to the domain gap between the artificially synthesized
and real LR images. Some efforts are thus taken to capture real-world image
pairs. The captured LR-HR image pairs usually suffer from unavoidable
misalignment, which hampers the performance of end-to-end learning, however.
Here, focusing on the real-world SR, we ask a different question: since
misalignment is unavoidable, can we propose a method that does not need LR-HR
image pairing and alignment at all and utilize real images as they are? Hence
we propose a framework to learn SR from an arbitrary set of unpaired LR and HR
images and see how far a step can go in such a realistic and "unsupervised"
setting. To do so, we firstly train a degradation generation network to
generate realistic LR images and, more importantly, to capture their
distribution (i.e., learning to zoom out). Instead of assuming the domain gap
has been eliminated, we minimize the discrepancy between the generated data and
real data while learning a degradation adaptive SR network (i.e., learning to
zoom in). The proposed unpaired method achieves state-of-the-art SR results on
real-world images, even in the datasets that favor the paired-learning methods
more.
