  The image Super-Resolution (SR) technique has greatly improved the visual
quality of images by enhancing their resolutions. It also calls for an
efficient SR Image Quality Assessment (SR-IQA) to evaluate those algorithms or
their generated images. In this paper, we focus on the SR-IQA under deep
learning and propose a Structure-and-Perception-based Quality Evaluation
(SPQE). In emerging deep-learning-based SR, a generated high-quality, visually
pleasing image may have different structures from its corresponding low-quality
image. In such case, how to balance the quality scores between no-reference
perceptual quality and referenced structural similarity is a critical issue. To
help ease this problem, we give a theoretical analysis on this tradeoff and
further calculate adaptive weights for the two types of quality scores. We also
propose two deep-learning-based regressors to model the no-reference and
referenced scores. By combining the quality scores and their weights, we
propose a unified SPQE metric for SR-IQA. Experimental results demonstrate that
the proposed method outperforms the state-of-the-arts in different datasets.
