  The representations of neural networks are often compared to those of
biological systems by performing regression between the neural network
responses and those measured from biological systems. Many different
state-of-the-art deep neural networks yield similar neural predictions, but it
remains unclear how to differentiate among models that perform equally well at
predicting neural responses. To gain insight into this, we use a recent
theoretical framework that relates the generalization error from regression to
the spectral properties of the model and the target. We apply this theory to
the case of regression between model activations and neural responses and
decompose the neural prediction error in terms of the model eigenspectra,
alignment of model eigenvectors and neural responses, and the training set
size. Using this decomposition, we introduce geometrical measures to interpret
the neural prediction error. We test a large number of deep neural networks
that predict visual cortical activity and show that there are multiple types of
geometries that result in low neural prediction error as measured via
regression. The work demonstrates that carefully decomposing representational
metrics can provide interpretability of how models are capturing neural
activity and points the way towards improved models of neural activity.
