  Structures matter in single image super resolution (SISR). Recent studies
benefiting from generative adversarial network (GAN) have promoted the
development of SISR by recovering photo-realistic images. However, there are
always undesired structural distortions in the recovered images. In this paper,
we propose a structure-preserving super resolution method to alleviate the
above issue while maintaining the merits of GAN-based methods to generate
perceptual-pleasant details. Specifically, we exploit gradient maps of images
to guide the recovery in two aspects. On the one hand, we restore
high-resolution gradient maps by a gradient branch to provide additional
structure priors for the SR process. On the other hand, we propose a gradient
loss which imposes a second-order restriction on the super-resolved images.
Along with the previous image-space loss functions, the gradient-space
objectives help generative networks concentrate more on geometric structures.
Moreover, our method is model-agnostic, which can be potentially used for
off-the-shelf SR networks. Experimental results show that we achieve the best
PI and LPIPS performance and meanwhile comparable PSNR and SSIM compared with
state-of-the-art perceptual-driven SR methods. Visual results demonstrate our
superiority in restoring structures while generating natural SR images.
