  We apply topological data analysis methods to loss functions to gain insights
into learning of deep neural networks and deep neural networks generalization
properties. We use the Morse complex of the loss function to relate the local
behavior of gradient descent trajectories with global properties of the loss
surface. We define the neural network Topological Obstructions score,
"TO-score", with the help of robust topological invariants, barcodes of the
loss function, that quantify the "badness" of local minima for gradient-based
optimization. We have made experiments for computing these invariants for
fully-connected, convolutional and ResNet-like neural networks on different
datasets: MNIST, Fashion MNIST, CIFAR10, CIFAR100 and SVHN. Our two principal
observations are as follows. Firstly, the neural network barcode and TO score
decrease with the increase of the neural network depth and width, thus the
topological obstructions to learning diminish. Secondly, in certain situations
there is an intriguing connection between the lengths of minima segments in the
barcode and the minima generalization errors.
