  While existing strategies to execute deep learning-based classification on
low-power platforms assume the models are trained on all classes of interest,
this paper posits that adopting context-awareness i.e. narrowing down a
classification task to the current deployment context consisting of only recent
inference queries can substantially enhance performance in resource-constrained
environments. We propose a new paradigm, CACTUS, for scalable and efficient
context-aware classification where a micro-classifier recognizes a small set of
classes relevant to the current context and, when context change happens (e.g.,
a new class comes into the scene), rapidly switches to another suitable
micro-classifier. CACTUS features several innovations, including optimizing the
training cost of context-aware classifiers, enabling on-the-fly context-aware
switching between classifiers, and balancing context switching costs and
performance gains via simple yet effective switching policies. We show that
CACTUS achieves significant benefits in accuracy, latency, and compute budget
across a range of datasets and IoT platforms.
