  This paper introduces ASCAI, a novel adaptive sampling methodology that can
learn how to effectively compress Deep Neural Networks (DNNs) for accelerated
inference on resource-constrained platforms. Modern DNN compression techniques
comprise various hyperparameters that require per-layer customization to ensure
high accuracy. Choosing such hyperparameters is cumbersome as the pertinent
search space grows exponentially with the number of model layers. To
effectively traverse this large space, we devise an intelligent sampling
mechanism that adapts the sampling strategy using customized operations
inspired by genetic algorithms. As a special case, we consider the space of
model compression as a vector space. The adaptively selected samples enable
ASCAI to automatically learn how to tune per-layer compression hyperparameters
to optimize the accuracy/model-size trade-off. Our extensive evaluations show
that ASCAI outperforms rule-based and reinforcement learning methods in terms
of compression rate and/or accuracy
