  Structured weight pruning is a representative model compression technique of
DNNs for hardware efficiency and inference accelerations. Previous works in
this area leave great space for improvement since sparse structures with
combinations of different structured pruning schemes are not exploited fully
and efficiently. To mitigate the limitations, we propose SS-Auto, a
single-shot, automatic structured pruning framework that can achieve row
pruning and column pruning simultaneously. We adopt soft constraint-based
formulation to alleviate the strong non-convexity of l0-norm constraints used
in state-of-the-art ADMM-based methods for faster convergence and fewer
hyperparameters. Instead of solving the problem directly, a Primal-Proximal
solution is proposed to avoid the pitfall of penalizing all weights equally,
thereby enhancing the accuracy. Extensive experiments on CIFAR-10 and CIFAR-100
datasets demonstrate that the proposed framework can achieve ultra-high pruning
rates while maintaining accuracy. Furthermore, significant inference speedup
has been observed from the proposed framework through actual measurements on
the smartphone.
