  Neural networks (NNs) are making a large impact both on research and
industry. Nevertheless, as NNs' accuracy increases, it is followed by an
expansion in their size, required number of compute operations and energy
consumption. Increase in resource consumption results in NNs' reduced adoption
rate and real-world deployment impracticality. Therefore, NNs need to be
compressed to make them available to a wider audience and at the same time
decrease their runtime costs. In this work, we approach this challenge from a
causal inference perspective, and we propose a scoring mechanism to facilitate
structured pruning of NNs. The approach is based on measuring mutual
information under a maximum entropy perturbation, sequentially propagated
through the NN. We demonstrate the method's performance on two datasets and
various NNs' sizes, and we show that our approach achieves competitive
performance under challenging conditions.
