  Partially inspired by features of computation in visual cortex, deep neural
networks compute hierarchical representations of their inputs. While these
networks have been highly successful in machine learning, it remains unclear to
what extent they can aid our understanding of cortical function. Several groups
have developed metrics that provide a quantitative comparison between
representations computed by networks and representations measured in cortex. At
the same time, neuroscience is well into an unprecedented phase of large-scale
data collection, as evidenced by projects such as the Allen Brain Observatory.
Despite the magnitude of these efforts, in a given experiment only a fraction
of units are recorded, limiting the information available about the cortical
representation. Moreover, only a finite number of stimuli can be shown to an
animal over the course of a realistic experiment. These limitations raise the
question of how and whether metrics that compare representations of deep
networks are meaningful on these datasets. Here, we empirically quantify the
capabilities and limitations of these metrics due to limited image
presentations and neuron samples. We find that the comparison procedure is
robust to different choices of stimuli set and the level of subsampling that
one might expect in a large-scale brain survey with thousands of neurons. Using
these results, we compare the representations measured in the Allen Brain
Observatory in response to natural image presentations to deep neural network.
We show that the visual cortical areas are relatively high order
representations (in that they map to deeper layers of convolutional neural
networks). Furthermore, we see evidence of a broad, more parallel organization
rather than a sequential hierarchy, with the primary area VISp(V1) being lower
order relative to the other areas.
