  In this paper we show the similarities and differences of two deep neural
networks by comparing the manifolds composed of activation vectors in each
fully connected layer of them. The main contribution of this paper includes 1)
a new data generating algorithm which is crucial for determining the dimension
of manifolds; 2) a systematic strategy to compare manifolds. Especially, we
take Riemann curvature and sectional curvature as part of criterion, which can
reflect the intrinsic geometric properties of manifolds. Some interesting
results and phenomenon are given, which help in specifying the similarities and
differences between the features extracted by two networks and demystifying the
intrinsic mechanism of deep neural networks.
