  Structured weight pruning is a representative model compression technique of
DNNs to reduce the storage and computation requirements and accelerate
inference. An automatic hyperparameter determination process is necessary due
to the large number of flexible hyperparameters. This work proposes
AutoCompress, an automatic structured pruning framework with the following key
performance improvements: (i) effectively incorporate the combination of
structured pruning schemes in the automatic process; (ii) adopt the
state-of-art ADMM-based structured weight pruning as the core algorithm, and
propose an innovative additional purification step for further weight reduction
without accuracy loss; and (iii) develop effective heuristic search method
enhanced by experience-based guided search, replacing the prior deep
reinforcement learning technique which has underlying incompatibility with the
target pruning problem. Extensive experiments on CIFAR-10 and ImageNet datasets
demonstrate that AutoCompress is the key to achieve ultra-high pruning rates on
the number of weights and FLOPs that cannot be achieved before. As an example,
AutoCompress outperforms the prior work on automatic model compression by up to
33x in pruning rate (120x reduction in the actual parameter count) under the
same accuracy. Significant inference speedup has been observed from the
AutoCompress framework on actual measurements on smartphone. We release all
models of this work at anonymous link: http://bit.ly/2VZ63dS.
