  Convolutional Neural Networks(CNNs) are both computation and memory intensive
which hindered their deployment in mobile devices. Inspired by the relevant
concept in neural science literature, we propose Synaptic Pruning: a
data-driven method to prune connections between input and output feature maps
with a newly proposed class of parameters called Synaptic Strength. Synaptic
Strength is designed to capture the importance of a connection based on the
amount of information it transports. Experiment results show the effectiveness
of our approach. On CIFAR-10, we prune connections for various CNN models with
up to 96% , which results in significant size reduction and computation saving.
Further evaluation on ImageNet demonstrates that synaptic pruning is able to
discover efficient models which is competitive to state-of-the-art compact CNNs
such as MobileNet-V2 and NasNet-Mobile. Our contribution is summarized as
following: (1) We introduce Synaptic Strength, a new class of parameters for
CNNs to indicate the importance of each connections. (2) Our approach can prune
various CNNs with high compression without compromising accuracy. (3) Further
investigation shows, the proposed Synaptic Strength is a better indicator for
kernel pruning compared with the previous approach in both empirical result and
theoretical analysis.
