  Deep neural networks progressively transform their inputs across multiple
processing layers. What are the geometrical properties of the representations
learned by these networks? Here we study the intrinsic dimensionality (ID) of
data-representations, i.e. the minimal number of parameters needed to describe
a representation. We find that, in a trained network, the ID is orders of
magnitude smaller than the number of units in each layer. Across layers, the ID
first increases and then progressively decreases in the final layers.
Remarkably, the ID of the last hidden layer predicts classification accuracy on
the test set. These results can neither be found by linear dimensionality
estimates (e.g., with principal component analysis), nor in representations
that had been artificially linearized. They are neither found in untrained
networks, nor in networks that are trained on randomized labels. This suggests
that neural networks that can generalize are those that transform the data into
low-dimensional, but not necessarily flat manifolds.
