  Nowadays, neural-network-based image- and video-quality metrics perform
better than traditional methods. However, they also became more vulnerable to
adversarial attacks that increase metrics' scores without improving visual
quality. The existing benchmarks of quality metrics compare their performance
in terms of correlation with subjective quality and calculation time.
Nonetheless, the adversarial robustness of image-quality metrics is also an
area worth researching. This paper analyses modern metrics' robustness to
different adversarial attacks. We adapted adversarial attacks from computer
vision tasks and compared attacks' efficiency against 15 no-reference image-
and video-quality metrics. Some metrics showed high resistance to adversarial
attacks, which makes their usage in benchmarks safer than vulnerable metrics.
The benchmark accepts submissions of new metrics for researchers who want to
make their metrics more robust to attacks or to find such metrics for their
needs. The latest results can be found online:
https://videoprocessing.ai/benchmarks/metrics-robustness.html.
