  Deep Neural Networks (DNNs) have been widely applied in Internet of Things
(IoT) systems for various tasks such as image classification and object
detection. However, heavyweight DNN models can hardly be deployed on edge
devices due to limited computational resources. In this paper, an edge-cloud
cooperation framework is proposed to improve inference accuracy while
maintaining low inference latency. To this end, we deploy a lightweight model
on the edge and a heavyweight model on the cloud. A reinforcement learning
(RL)-based DNN compression approach is used to generate the lightweight model
suitable for the edge from the heavyweight model. Moreover, a supervised
learning (SL)-based offloading strategy is applied to determine whether the
sample should be processed on the edge or on the cloud. Our method is
implemented on real hardware and tested on multiple datasets. The experimental
results show that (1) The sizes of the lightweight models obtained by RL-based
DNN compression are up to 87.6% smaller than those obtained by the baseline
method; (2) SL-based offloading strategy makes correct offloading decisions in
most cases; (3) Our method reduces up to 78.8% inference latency and achieves
higher accuracy compared with the cloud-only strategy.
