  Deep neural networks (DNNs) are usually over-parameterized to increase the
likelihood of getting adequate initial weights by random initialization.
Consequently, trained DNNs have many redundancies which can be pruned from the
model to reduce complexity and improve the ability to generalize. Structural
sparsity, as achieved by filter pruning, directly reduces the tensor sizes of
weights and activations and is thus particularly effective for reducing
complexity. We propose "Holistic Filter Pruning" (HFP), a novel approach for
common DNN training that is easy to implement and enables to specify accurate
pruning rates for the number of both parameters and multiplications. After each
forward pass, the current model complexity is calculated and compared to the
desired target size. By gradient descent, a global solution can be found that
allocates the pruning budget over the individual layers such that the desired
target size is fulfilled. In various experiments, we give insights into the
training and achieve state-of-the-art performance on CIFAR-10 and ImageNet (HFP
prunes 60% of the multiplications of ResNet-50 on ImageNet with no significant
loss in the accuracy). We believe our simple and powerful pruning approach to
constitute a valuable contribution for users of DNNs in low-cost applications.
