  The success of deep neural networks in many real-world applications is
leading to new challenges in building more efficient architectures. One
effective way of making networks more efficient is neural network compression.
We provide an overview of existing neural network compression methods that can
be used to make neural networks more efficient by changing the architecture of
the network. First, we introduce a new way to categorize all published
compression methods, based on the amount of data and compute needed to make the
methods work in practice. These are three 'levels of compression solutions'.
Second, we provide a taxonomy of tensor factorization based and probabilistic
compression methods. Finally, we perform an extensive evaluation of different
compression techniques from the literature for models trained on ImageNet. We
show that SVD and probabilistic compression or pruning methods are
complementary and give the best results of all the considered methods. We also
provide practical ways to combine them.
