  Modern deep neural network models are large and computationally intensive.
One typical solution to this issue is model pruning. However, most current
pruning algorithms depend on hand crafted rules or domain expertise. To
overcome this problem, we propose a learning based auto pruning algorithm for
deep neural network, which is inspired by recent automatic machine
learning(AutoML). A two objectives' problem that aims for the the weights and
the best channels for each layer is first formulated. An alternative
optimization approach is then proposed to derive the optimal channel numbers
and weights simultaneously. In the process of pruning, we utilize a searchable
hyperparameter, remaining ratio, to denote the number of channels in each
convolution layer, and then a dynamic masking process is proposed to describe
the corresponding channel evolution. To control the trade-off between the
accuracy of a model and the pruning ratio of floating point operations, a novel
loss function is further introduced. Preliminary experimental results on
benchmark datasets demonstrate that our scheme achieves competitive results for
neural network pruning.
