  Compressing convolutional neural networks (CNNs) by pruning and distillation
has received ever-increasing focus in the community. In particular, designing a
class-discrimination based approach would be desired as it fits seamlessly with
the CNNs training objective. In this paper, we propose class-discriminative
compression (CDC), which injects class discrimination in both pruning and
distillation to facilitate the CNNs training goal. We first study the
effectiveness of a group of discriminant functions for channel pruning, where
we include well-known single-variate binary-class statistics like Student's
T-Test in our study via an intuitive generalization. We then propose a novel
layer-adaptive hierarchical pruning approach, where we use a coarse class
discrimination scheme for early layers and a fine one for later layers. This
method naturally accords with the fact that CNNs process coarse semantics in
the early layers and extract fine concepts at the later. Moreover, we leverage
discriminant component analysis (DCA) to distill knowledge of intermediate
representations in a subspace with rich discriminative information, which
enhances hidden layers' linear separability and classification accuracy of the
student. Combining pruning and distillation, CDC is evaluated on CIFAR and
ILSVRC 2012, where we consistently outperform the state-of-the-art results.
