  With the remarkable success of deep learning recently, efficient network
compression algorithms are urgently demanded for releasing the potential
computational power of edge devices, such as smartphones or tablets. However,
optimal network pruning is a non-trivial task which mathematically is an
NP-hard problem. Previous researchers explain training a pruned network as
buying a lottery ticket. In this paper, we investigate the Magnitude-Based
Pruning (MBP) scheme and analyze it from a novel perspective through Fourier
analysis on the deep learning model to guide model designation. Besides
explaining the generalization ability of MBP using Fourier transform, we also
propose a novel two-stage pruning approach, where one stage is to obtain the
topological structure of the pruned network and the other stage is to retrain
the pruned network to recover the capacity using knowledge distillation from
lower to higher on the frequency domain. Extensive experiments on CIFAR-10 and
CIFAR-100 demonstrate the superiority of our novel Fourier analysis based MBP
compared to other traditional MBP algorithms.
