  Measuring the similarity of images is a fundamental problem to computer
vision for which no universal solution exists. While simple metrics such as the
pixel-wise L2-norm have been shown to have significant flaws, they remain
popular. One group of recent state-of-the-art metrics that mitigates some of
those flaws are Deep Perceptual Similarity (DPS) metrics, where the similarity
is evaluated as the distance in the deep features of neural networks. However,
DPS metrics themselves have been less thoroughly examined for their benefits
and, especially, their flaws. This work investigates the most common DPS
metric, where deep features are compared by spatial position, along with
metrics comparing the averaged and sorted deep features. The metrics are
analyzed in-depth to understand the strengths and weaknesses of the metrics by
using images designed specifically to challenge them. This work contributes
with new insights into the flaws of DPS, and further suggests improvements to
the metrics. An implementation of this work is available online:
https://github.com/guspih/deep_perceptual_similarity_analysis/
