  Recent advances in deep learning motivate the use of deep neutral networks in
sensing applications, but their excessive resource needs on constrained
embedded devices remain an important impediment. A recently explored solution
space lies in compressing (approximating or simplifying) deep neural networks
in some manner before use on the device. We propose a new compression solution,
called DeepIoT, that makes two key contributions in that space. First, unlike
current solutions geared for compressing specific types of neural networks,
DeepIoT presents a unified approach that compresses all commonly used deep
learning structures for sensing applications, including fully-connected,
convolutional, and recurrent neural networks, as well as their combinations.
Second, unlike solutions that either sparsify weight matrices or assume linear
structure within weight matrices, DeepIoT compresses neural network structures
into smaller dense matrices by finding the minimum number of non-redundant
hidden elements, such as filters and dimensions required by each layer, while
keeping the performance of sensing applications the same. Importantly, it does
so using an approach that obtains a global view of parameter redundancies,
which is shown to produce superior compression. We conduct experiments with
five different sensing-related tasks on Intel Edison devices. DeepIoT
outperforms all compared baseline algorithms with respect to execution time and
energy consumption by a significant margin. It reduces the size of deep neural
networks by 90% to 98.9%. It is thus able to shorten execution time by 71.4% to
94.5%, and decrease energy consumption by 72.2% to 95.7%. These improvements
are achieved without loss of accuracy. The results underscore the potential of
DeepIoT for advancing the exploitation of deep neural networks on
resource-constrained embedded devices.
