  Many Internet-of-Things (IoT) applications demand fast and accurate
understanding of a few key events in their surrounding environment. Deep
Convolutional Neural Networks (CNNs) have emerged as an effective approach to
understand speech, images, and similar high dimensional data types. Algorithmic
performance of modern CNNs, however, fundamentally relies on learning
class-agnostic hierarchical features that only exist in comprehensive training
datasets with many classes. As a result, fast inference using CNNs trained on
such datasets is prohibitive for most resource-constrained IoT platforms. To
bridge this gap, we present a principled and practical methodology for
distilling a complex modern CNN that is trained to effectively recognize many
different classes of input data into an application-dependent essential core
that not only recognizes the few classes of interest to the application
accurately, but also runs efficiently on platforms with limited resources.
Experimental results confirm that our approach strikes a favorable balance
between classification accuracy (application constraint), inference efficiency
(platform constraint), and productive development of new applications (business
constraint).
