  Gradient descent during the learning process of a neural network can be
subject to many instabilities. The spectral density of the Jacobian is a key
component for analyzing stability. Following the works of Pennington et al.,
such Jacobians are modeled using free multiplicative convolutions from Free
Probability Theory (FPT).
  We present a reliable and very fast method for computing the associated
spectral densities, for given architecture and initialization. This method has
a controlled and proven convergence. Our technique is based on an homotopy
method: it is an adaptative Newton-Raphson scheme which chains basins of
attraction.
  In order to demonstrate the relevance of our method we show that the relevant
FPT metrics computed before training are highly correlated to final test
accuracies - up to 85\%. We also nuance the idea that learning happens at the
edge of chaos by giving evidence that a very desirable feature for neural
networks is the hyperbolicity of their Jacobian at initialization.
