  Reducing the test time resource requirements of a neural network while
preserving test accuracy is crucial for running inference on
resource-constrained devices. To achieve this goal, we introduce a novel
network reparameterization based on the Kronecker-factored eigenbasis (KFE),
and then apply Hessian-based structured pruning methods in this basis. As
opposed to existing Hessian-based pruning algorithms which do pruning in
parameter coordinates, our method works in the KFE where different weights are
approximately independent, enabling accurate pruning and fast computation. We
demonstrate empirically the effectiveness of the proposed method through
extensive experiments. In particular, we highlight that the improvements are
especially significant for more challenging datasets and networks. With
negligible loss of accuracy, an iterative-pruning version gives a 10$\times$
reduction in model size and a 8$\times$ reduction in FLOPs on wide ResNet32.
