  Image Quality Assessment (IQA) is a fundamental task in computer vision that
has witnessed remarkable progress with deep neural networks. Inspired by the
characteristics of the human visual system, existing methods typically use a
combination of global and local representations (\ie, multi-scale features) to
achieve superior performance. However, most of them adopt simple linear fusion
of multi-scale features, and neglect their possibly complex relationship and
interaction. In contrast, humans typically first form a global impression to
locate important regions and then focus on local details in those regions. We
therefore propose a top-down approach that uses high-level semantics to guide
the IQA network to focus on semantically important local distortion regions,
named as \emph{TOPIQ}. Our approach to IQA involves the design of a heuristic
coarse-to-fine network (CFANet) that leverages multi-scale features and
progressively propagates multi-level semantic information to low-level
representations in a top-down manner. A key component of our approach is the
proposed cross-scale attention mechanism, which calculates attention maps for
lower level features guided by higher level features. This mechanism emphasizes
active semantic regions for low-level distortions, thereby improving
performance. CFANet can be used for both Full-Reference (FR) and No-Reference
(NR) IQA. We use ResNet50 as its backbone and demonstrate that CFANet achieves
better or competitive performance on most public FR and NR benchmarks compared
with state-of-the-art methods based on vision transformers, while being much
more efficient (with only ${\sim}13\%$ FLOPS of the current best FR method).
Codes are released at \url{https://github.com/chaofengc/IQA-PyTorch}.
