  Understanding the black-box representations in Deep Neural Networks (DNN) is
an essential problem in deep learning. In this work, we propose Graph-Based
Similarity (GBS) to measure the similarity of layer features. Contrary to
previous works that compute the similarity directly on the feature maps, GBS
measures the correlation based on the graph constructed with hidden layer
outputs. By treating each input sample as a node and the corresponding layer
output similarity as edges, we construct the graph of DNN representations for
each layer. The similarity between graphs of layers identifies the
correspondences between representations of models trained in different datasets
and initializations. We demonstrate and prove the invariance property of GBS,
including invariance to orthogonal transformation and invariance to isotropic
scaling, and compare GBS with CKA. GBS shows state-of-the-art performance in
reflecting the similarity and provides insights on explaining the adversarial
sample behavior on the hidden layer space.
