  Existing differentiable channel pruning methods often attach scaling factors
or masks behind channels to prune filters with less importance, and implicitly
assume uniform contribution of input samples to filter importance.
Specifically, the effects of instance complexity on pruning performance are not
yet fully investigated in static network pruning. In this paper, we propose a
simple yet effective differentiable network pruning method CWP based on
instance complexity weighted filter importance scores. We define instance
complexity related weight for each instance by giving higher weights to hard
instances, and measure the weighted sum of instance-specific soft masks to
model non-uniform contribution of different inputs, which encourages hard
instances to dominate the pruning process and the model performance to be well
preserved. In addition, we introduce a regularizer to maximize polarization of
the masks, such that a sweet spot can be easily found to identify the filters
to be pruned. Performance evaluations on various network architectures and
datasets demonstrate CWP has advantages over the state-of-the-arts in pruning
large networks. For instance, CWP improves the accuracy of ResNet56 on CIFAR-10
dataset by 0.32% aftering removing 64.11% FLOPs, and prunes 87.75% FLOPs of
ResNet50 on ImageNet dataset with only 0.93% Top-1 accuracy loss.
