  Runtime-tunable context-dependent network compression would make mobile deep
learning (DL) adaptable to often varying resource availability, input
"difficulty", or user needs. The existing compression techniques significantly
reduce the memory, processing, and energy tax of DL, yet, the resulting models
tend to be permanently impaired, sacrificing the inference power for reduced
resource usage. The existing tunable compression approaches, on the other hand,
require expensive re-training, do not support arbitrary strategies for adapting
the compression and do not provide mobile-ready implementations.
  In this paper we present Mobiprox, a framework enabling mobile DL with
flexible precision. Mobiprox implements tunable approximations of tensor
operations and enables runtime-adaptable approximation of individual network
layers. A profiler and a tuner included with Mobiprox identify the most
promising neural network approximation configurations leading to the desired
inference quality with the minimal use of resources. Furthermore, we develop
control strategies that depending on contextual factors, such as the input data
difficulty, dynamically adjust the approximation levels across a mobile DL
model's layers. We implement Mobiprox in Android OS and through experiments in
diverse mobile domains, including human activity recognition and spoken keyword
detection, demonstrate that it can save up to 15% system-wide energy with a
minimal impact on the inference accuracy.
