  Image quality assessment (IQA) algorithm aims to quantify the human
perception of image quality. Unfortunately, there is a performance drop when
assessing the distortion images generated by generative adversarial network
(GAN) with seemingly realistic texture. In this work, we conjecture that this
maladaptation lies in the backbone of IQA models, where patch-level prediction
methods use independent image patches as input to calculate their scores
separately, but lack spatial relationship modeling among image patches.
Therefore, we propose an Attention-based Hybrid Image Quality Assessment
Network (AHIQ) to deal with the challenge and get better performance on the
GAN-based IQA task. Firstly, we adopt a two-branch architecture, including a
vision transformer (ViT) branch and a convolutional neural network (CNN) branch
for feature extraction. The hybrid architecture combines interaction
information among image patches captured by ViT and local texture details from
CNN. To make the features from shallow CNN more focused on the visually salient
region, a deformable convolution is applied with the help of semantic
information from the ViT branch. Finally, we use a patch-wise score prediction
module to obtain the final score. The experiments show that our model
outperforms the state-of-the-art methods on four standard IQA datasets and AHIQ
ranked first on the Full Reference (FR) track of the NTIRE 2022 Perceptual
Image Quality Assessment Challenge.
