  Budgeted pruning is the problem of pruning under resource constraints. In
budgeted pruning, how to distribute the resources across layers (i.e., sparsity
allocation) is the key problem. Traditional methods solve it by discretely
searching for the layer-wise pruning ratios, which lacks efficiency. In this
paper, we propose Differentiable Sparsity Allocation (DSA), an efficient
end-to-end budgeted pruning flow. Utilizing a novel differentiable pruning
process, DSA finds the layer-wise pruning ratios with gradient-based
optimization. It allocates sparsity in continuous space, which is more
efficient than methods based on discrete evaluation and search. Furthermore,
DSA could work in a pruning-from-scratch manner, whereas traditional budgeted
pruning methods are applied to pre-trained models. Experimental results on
CIFAR-10 and ImageNet show that DSA could achieve superior performance than
current iterative budgeted pruning methods, and shorten the time cost of the
overall pruning process by at least 1.5x in the meantime.
