  Deep Neural Networks (DNNs) often fail in out-of-distribution scenarios. In
this paper, we introduce a tool to visualize and understand such failures. We
draw inspiration from concepts from neural electrophysiology, which are based
on inspecting the internal functioning of a neural networks by analyzing the
feature tuning and invariances of individual units. Deep Electrophysiology, in
short Deephys, provides insights of the DNN's failures in out-of-distribution
scenarios by comparative visualization of the neural activity in
in-distribution and out-of-distribution datasets. Deephys provides seamless
analyses of individual neurons, individual images, and a set of set of images
from a category, and it is capable of revealing failures due to the presence of
spurious features and novel features. We substantiate the validity of the
qualitative visualizations of Deephys thorough quantitative analyses using
convolutional and transformers architectures, in several datasets and
distribution shifts (namely, colored MNIST, CIFAR-10 and ImageNet).
