  Convolutional Neural Network (CNN) based Deep Learning (DL) has achieved
great progress in many real-life applications. Meanwhile, due to the complex
model structures against strict latency and memory restriction, the
implementation of CNN models on the resource-limited platforms is becoming more
challenging. This work proposes a solution, called CompactNet\footnote{Project
URL: \url{https://github.com/CompactNet/CompactNet}}, which automatically
optimizes a pre-trained CNN model on a specific resource-limited platform given
a specific target of inference speedup. Guided by a simulator of the target
platform, CompactNet progressively trims a pre-trained network by removing
certain redundant filters until the target speedup is reached and generates an
optimal platform-specific model while maintaining the accuracy. We evaluate our
work on two platforms of a mobile ARM CPU and a machine learning accelerator
NPU (Cambricon-1A ISA) on a Huawei Mate10 smartphone. For the state-of-the-art
slim CNN model made for the embedded platform, MobileNetV2, CompactNet achieves
up to a 1.8x kernel computation speedup with equal or even higher accuracy for
image classification tasks on the Cifar-10 dataset.
