  Reducing the high computational cost of large convolutional neural networks
is crucial when deploying the networks to resource-constrained environments. We
first show the greedy approach of recent channel pruning methods ignores the
inherent quadratic coupling between channels in the neighboring layers and
cannot safely remove inactive weights during the pruning procedure.
Furthermore, due to these inactive weights, the greedy methods cannot guarantee
to satisfy the given resource constraints and deviate with the true objective.
In this regard, we propose a novel channel selection method that optimally
selects channels via discrete QCQP, which provably prevents any inactive
weights and guarantees to meet the resource constraints tightly in terms of
FLOPs, memory usage, and network size. We also propose a quadratic model that
accurately estimates the actual inference time of the pruned network, which
allows us to adopt inference time as a resource constraint option. Furthermore,
we generalize our method to extend the selection granularity beyond channels
and handle non-sequential connections. Our experiments on CIFAR-10 and ImageNet
show our proposed pruning method outperforms other fixed-importance channel
pruning methods on various network architectures.
