  Many attempts have been done to extend the great success of convolutional
neural networks (CNNs) achieved on high-end GPU servers to portable devices
such as smart phones. Providing compression and acceleration service of deep
learning models on the cloud is therefore of significance and is attractive for
end users. However, existing network compression and acceleration approaches
usually fine-tuning the svelte model by requesting the entire original training
data (\eg ImageNet), which could be more cumbersome than the network itself and
cannot be easily uploaded to the cloud. In this paper, we present a novel
positive-unlabeled (PU) setting for addressing this problem. In practice, only
a small portion of the original training set is required as positive examples
and more useful training examples can be obtained from the massive unlabeled
data on the cloud through a PU classifier with an attention based multi-scale
feature extractor. We further introduce a robust knowledge distillation (RKD)
scheme to deal with the class imbalance problem of these newly augmented
training examples. The superiority of the proposed method is verified through
experiments conducted on the benchmark models and datasets. We can use only
$8\%$ of uniformly selected data from the ImageNet to obtain an efficient model
with comparable performance to the baseline ResNet-34.
