  Quantifying the perceptual similarity of two images is a long-standing
problem in low-level computer vision. The natural image domain commonly relies
on supervised learning, e.g., a pre-trained VGG, to obtain a latent
representation. However, due to domain shift, pre-trained models from the
natural image domain might not apply to other image domains, such as medical
imaging. Notably, in medical imaging, evaluating the perceptual similarity is
exclusively performed by specialists trained extensively in diverse medical
fields. Thus, medical imaging remains devoid of task-specific, objective
perceptual measures. This work answers the question: Is it necessary to rely on
supervised learning to obtain an effective representation that could measure
perceptual similarity, or is self-supervision sufficient? To understand whether
recent contrastive self-supervised representation (CSR) may come to the rescue,
we start with natural images and systematically evaluate CSR as a metric across
numerous contemporary architectures and tasks and compare them with existing
methods. We find that in the natural image domain, CSR behaves on par with the
supervised one on several perceptual tests as a metric, and in the medical
domain, CSR better quantifies perceptual similarity concerning the experts'
ratings. We also demonstrate that CSR can significantly improve image quality
in two image synthesis tasks. Finally, our extensive results suggest that
perceptuality is an emergent property of CSR, which can be adapted to many
image domains without requiring annotations.
