  Deep neural networks use multiple layers of functions to map an object
represented by an input vector progressively to different representations, and
with sufficient training, eventually to a single score for each class that is
the output of the final decision function. Ideally, in this output space, the
objects of different classes achieve maximum separation. Motivated by the need
to better understand the inner working of a deep neural network, we analyze the
effectiveness of the learned representations in separating the classes from a
data complexity perspective. Using a simple complexity measure, a popular
benchmarking task, and a well-known architecture design, we show how the data
complexity evolves through the network, how it changes during training, and how
it is impacted by the network design and the availability of training samples.
We discuss the implications of the observations and the potentials for further
studies.
