  Pruning well-trained neural networks is effective to achieve a promising
accuracy-efficiency trade-off in computer vision regimes. However, most of
existing pruning algorithms only focus on the classification task defined on
the source domain. Different from the strong transferability of the original
model, a pruned network is hard to transfer to complicated downstream tasks
such as object detection arXiv:arch-ive/2012.04643. In this paper, we show that
the image-level pretrain task is not capable of pruning models for diverse
downstream tasks. To mitigate this problem, we introduce image reconstruction,
a pixel-level task, into the traditional pruning framework. Concretely, an
autoencoder is trained based on the original model, and then the pruning
process is optimized with both autoencoder and classification losses. The
empirical study on benchmark downstream tasks shows that the proposed method
can outperform state-of-the-art results explicitly.
