  Image quality assessment (IQA) algorithms aim to reproduce the human's
perception of the image quality. The growing popularity of image enhancement,
generation, and recovery models instigated the development of many methods to
assess their performance. However, most IQA solutions are designed to predict
image quality in the general domain, with the applicability to specific areas,
such as medical imaging, remaining questionable. Moreover, the selection of
these IQA metrics for a specific task typically involves intentionally induced
distortions, such as manually added noise or artificial blurring; yet, the
chosen metrics are then used to judge the output of real-life computer vision
models. In this work, we aspire to fill these gaps by carrying out the most
extensive IQA evaluation study for Magnetic Resonance Imaging (MRI) to date
(14,700 subjective scores). We use outputs of neural network models trained to
solve problems relevant to MRI, including image reconstruction in the scan
acceleration, motion correction, and denoising. Our emphasis is on reflecting
the radiologist's perception of the reconstructed images, gauging the most
diagnostically influential criteria for the quality of MRI scans:
signal-to-noise ratio, contrast-to-noise ratio, and the presence of artifacts.
Seven trained radiologists assess these distorted images, with their verdicts
then correlated with 35 different image quality metrics (full-reference,
no-reference, and distribution-based metrics considered). The top performers --
DISTS, HaarPSI, VSI, and FID-VGG16 -- are found to be efficient across three
proposed quality criteria, for all considered anatomies and the target tasks.
