  Despite the remarkable similarities between convolutional neural networks
(CNN) and the human brain, CNNs still fall behind humans in many visual tasks,
indicating that there still exist considerable differences between the two
systems. Here, we leverage adversarial noise (AN) and adversarial interference
(AI) images to quantify the consistency between neural representations and
perceptual outcomes in the two systems. Humans can successfully recognize AI
images as corresponding categories but perceive AN images as meaningless noise.
In contrast, CNNs can correctly recognize AN images but mistakenly classify AI
images into wrong categories with surprisingly high confidence. We use
functional magnetic resonance imaging to measure brain activity evoked by
regular and adversarial images in the human brain, and compare it to the
activity of artificial neurons in a prototypical CNN-AlexNet. In the human
brain, we find that the representational similarity between regular and
adversarial images largely echoes their perceptual similarity in all early
visual areas. In AlexNet, however, the neural representations of adversarial
images are inconsistent with network outputs in all intermediate processing
layers, providing no neural foundations for perceptual similarity. Furthermore,
we show that voxel-encoding models trained on regular images can successfully
generalize to the neural responses to AI images but not AN images. These
remarkable differences between the human brain and AlexNet in the
representation-perception relation suggest that future CNNs should emulate both
behavior and the internal neural presentations of the human brain.
