  Adaptive network pruning approach has recently drawn significant attention
due to its excellent capability to identify the importance and redundancy of
layers and filters and customize a suitable pruning solution. However, it
remains unsatisfactory since current adaptive pruning methods rely mostly on an
additional monitor to score layer and filter importance, and thus faces high
complexity and weak interpretability. To tackle these issues, we have deeply
researched the weight reconstruction process in iterative prune-train process
and propose a Protective Self-Adaptive Pruning (PSAP) method. First of all,
PSAP can utilize its own information, weight sparsity ratio, to adaptively
adjust pruning ratio of layers before each pruning step. Moreover, we propose a
protective reconstruction mechanism to prevent important filters from being
pruned through supervising gradients and to avoid unrecoverable information
loss as well. Our PSAP is handy and explicit because it merely depends on
weights and gradients of model itself, instead of requiring an additional
monitor as in early works. Experiments on ImageNet and CIFAR-10 also
demonstrate its superiority to current works in both accuracy and compression
ratio, especially for compressing with a high ratio or pruning from scratch.
