  Energy-efficient machine learning models that can run directly on edge
devices are of great interest in IoT applications, as they can reduce network
pressure and response latency, and improve privacy. An effective way to obtain
energy-efficiency with small accuracy drops is to sequentially execute a set of
increasingly complex models, early-stopping the procedure for "easy" inputs
that can be confidently classified by the smallest models. As a stopping
criterion, current methods employ a single threshold on the output
probabilities produced by each model. In this work, we show that such a
criterion is sub-optimal for datasets that include classes of different
complexity, and we demonstrate a more general approach based on per-classes
thresholds. With experiments on a low-power end-node, we show that our method
can significantly reduce the energy consumption compared to the
single-threshold approach.
