  It has been demonstrated many times that the behavior of the human visual
system is connected to the statistics of natural images. Since machine learning
relies on the statistics of training data as well, the above connection has
interesting implications when using perceptual distances (which mimic the
behavior of the human visual system) as a loss function. In this paper, we aim
to unravel the non-trivial relationships between the probability distribution
of the data, perceptual distances, and unsupervised machine learning. To this
end, we show that perceptual sensitivity is correlated with the probability of
an image in its close neighborhood. We also explore the relation between
distances induced by autoencoders and the probability distribution of the
training data, as well as how these induced distances are correlated with human
perception. Finally, we find perceptual distances do not always lead to
noticeable gains in performance over Euclidean distance in common image
processing tasks, except when data is scarce and the perceptual distance
provides regularization. We propose this may be due to a \emph{double-counting}
effect of the image statistics, once in the perceptual distance and once in the
training procedure.
