  Recent breakthroughs in Deep Neural Networks (DNNs) have fueled a
tremendously growing demand for bringing DNN-powered intelligence into mobile
platforms. While the potential of deploying DNNs on resource-constrained
platforms has been demonstrated by DNN compression techniques, the current
practice suffers from two limitations: 1) merely stand-alone compression
schemes are investigated even though each compression technique only suit for
certain types of DNN layers; and 2) mostly compression techniques are optimized
for DNNs' inference accuracy, without explicitly considering other
application-driven system performance (e.g., latency and energy cost) and the
varying resource availability across platforms (e.g., storage and processing
capability). To this end, we propose AdaDeep, a usage-driven, automated DNN
compression framework for systematically exploring the desired trade-off
between performance and resource constraints, from a holistic system level.
Specifically, in a layer-wise manner, AdaDeep automatically selects the most
suitable combination of compression techniques and the corresponding
compression hyperparameters for a given DNN. Thorough evaluations on six
datasets and across twelve devices demonstrate that AdaDeep can achieve up to
$18.6\times$ latency reduction, $9.8\times$ energy-efficiency improvement, and
$37.3\times$ storage reduction in DNNs while incurring negligible accuracy
loss. Furthermore, AdaDeep also uncovers multiple novel combinations of
compression techniques.
