  In this paper, a simple yet effective network pruning framework is proposed
to simultaneously address the problems of pruning indicator, pruning ratio, and
efficiency constraint. This paper argues that the pruning decision should
depend on the convolutional weights, and thus proposes novel weight-dependent
gates (W-Gates) to learn the information from filter weights and obtain binary
gates to prune or keep the filters automatically. To prune the network under
efficiency constraints, a switchable Efficiency Module is constructed to
predict the hardware latency or FLOPs of candidate pruned networks. Combined
with the proposed Efficiency Module, W-Gates can perform filter pruning in an
efficiency-aware manner and achieve a compact network with a better
accuracy-efficiency trade-off. We have demonstrated the effectiveness of the
proposed method on ResNet34, ResNet50, and MobileNet V2, respectively achieving
up to 1.33/1.28/1.1 higher Top-1 accuracy with lower hardware latency on
ImageNet. Compared with state-of-the-art methods, W-Gates also achieves
superior performance.
