  Pruning is a promising approach to compress complex deep learning models in
order to deploy them on resource-constrained edge devices. However, many
existing pruning solutions are based on unstructured pruning, which yields
models that cannot efficiently run on commodity hardware and require users to
manually explore and tune the pruning process, which is time-consuming and
often leads to sub-optimal results. To address these limitations, this paper
presents an adaptive, activation-based, structured pruning approach to
automatically and efficiently generate small, accurate, and hardware-efficient
models that meet user requirements. First, it proposes iterative structured
pruning using activation-based attention feature maps to effectively identify
and prune unimportant filters. Then, it proposes adaptive pruning policies for
automatically meeting the pruning objectives of accuracy-critical,
memory-constrained, and latency-sensitive tasks. A comprehensive evaluation
shows that the proposed method can substantially outperform the
state-of-the-art structured pruning works on CIFAR-10 and ImageNet datasets.
For example, on ResNet-56 with CIFAR-10, without any accuracy drop, our method
achieves the largest parameter reduction (79.11%), outperforming the related
works by 22.81% to 66.07%, and the largest FLOPs reduction (70.13%),
outperforming the related works by 14.13% to 26.53%.
