  Post-training Sparsity (PTS) is a recently emerged avenue that chases
efficient network sparsity with limited data in need. Existing PTS methods,
however, undergo significant performance degradation compared with traditional
methods that retrain the sparse networks via the whole dataset, especially at
high sparsity ratios. In this paper, we attempt to reconcile this disparity by
transposing three cardinal factors that profoundly alter the performance of
conventional sparsity into the context of PTS. Our endeavors particularly
comprise (1) A base-decayed sparsity objective that promotes efficient
knowledge transferring from dense network to the sparse counterpart. (2) A
reducing-regrowing search algorithm designed to ascertain the optimal sparsity
distribution while circumventing overfitting to the small calibration set in
PTS. (3) The employment of dynamic sparse training predicated on the preceding
aspects, aimed at comprehensively optimizing the sparsity structure while
ensuring training stability. Our proposed framework, termed UniPTS, is
validated to be much superior to existing PTS methods across extensive
benchmarks. As an illustration, it amplifies the performance of POT, a recently
proposed recipe, from 3.9% to 68.6% when pruning ResNet-50 at 90% sparsity
ratio on ImageNet. We release the code of our paper at
https://github.com/xjjxmu/UniPTS.
