  Consistency and realness have always been the two critical issues of image
super-resolution. While the realness has been dramatically improved with the
use of GAN prior, the state-of-the-art methods still suffer inconsistencies in
local structures and colors (e.g., tooth and eyes). In this paper, we show that
these inconsistencies can be analytically eliminated by learning only the
null-space component while fixing the range-space part. Further, we design a
pooling-based decomposition (PD), a universal range-null space decomposition
for super-resolution tasks, which is concise, fast, and parameter-free. PD can
be easily applied to state-of-the-art GAN Prior based SR methods to eliminate
their inconsistencies, neither compromising the realness nor bringing extra
parameters or computational costs. Besides, our ablation studies reveal that PD
can replace pixel-wise losses for training and achieve better generalization
performance when facing unseen downsamplings or even real-world degradation.
Experiments show that the use of PD refreshes state-of-the-art SR performance
and speeds up the convergence of training up to 2~10 times.
