  It is widely believed that learning good representations is one of the main
reasons for the success of deep neural networks. Although highly intuitive,
there is a lack of theory and systematic approach quantitatively characterizing
what representations do deep neural networks learn. In this work, we move a
tiny step towards a theory and better understanding of the representations.
Specifically, we study a simpler problem: How similar are the representations
learned by two networks with identical architecture but trained from different
initializations. We develop a rigorous theory based on the neuron activation
subspace match model. The theory gives a complete characterization of the
structure of neuron activation subspace matches, where the core concepts are
maximum match and simple match which describe the overall and the finest
similarity between sets of neurons in two networks respectively. We also
propose efficient algorithms to find the maximum match and simple matches.
Finally, we conduct extensive experiments using our algorithms. Experimental
results suggest that, surprisingly, representations learned by the same
convolutional layers of networks trained from different initializations are not
as similar as prevalently expected, at least in terms of subspace match.
