  Deep neural networks have exhibited promising performance in image
super-resolution (SR) by learning a nonlinear mapping function from
low-resolution (LR) images to high-resolution (HR) images. However, there are
two underlying limitations to existing SR methods. First, learning the mapping
function from LR to HR images is typically an ill-posed problem, because there
exist infinite HR images that can be downsampled to the same LR image. As a
result, the space of the possible functions can be extremely large, which makes
it hard to find a good solution. Second, the paired LR-HR data may be
unavailable in real-world applications and the underlying degradation method is
often unknown. For such a more general case, existing SR models often incur the
adaptation problem and yield poor performance. To address the above issues, we
propose a dual regression scheme by introducing an additional constraint on LR
data to reduce the space of the possible functions. Specifically, besides the
mapping from LR to HR images, we learn an additional dual regression mapping
estimates the down-sampling kernel and reconstruct LR images, which forms a
closed-loop to provide additional supervision. More critically, since the dual
regression process does not depend on HR images, we can directly learn from LR
images. In this sense, we can easily adapt SR models to real-world data, e.g.,
raw video frames from YouTube. Extensive experiments with paired training data
and unpaired real-world data demonstrate our superiority over existing methods.
