  We explore the applications of random matrix theory (RMT) in the training of
deep neural networks (DNNs), focusing on layer pruning that is reducing the
number of DNN parameters (weights). Our numerical results show that this
pruning leads to a drastic reduction of parameters while not reducing the
accuracy of DNNs and CNNs. Moreover, pruning the fully connected DNNs actually
increases the accuracy and decreases the variance for random initializations.
Our numerics indicate that this enhancement in accuracy is due to the
simplification of the loss landscape. We next provide rigorous mathematical
underpinning of these numerical results by proving the RMT-based Pruning
Theorem. Our results offer valuable insights into the practical application of
RMT for the creation of more efficient and accurate deep-learning models.
