  IoT devices are powered by microcontroller units (MCUs) which are extremely
resource-scarce: a typical MCU may have an underpowered processor and around 64
KB of memory and persistent storage, which is orders of magnitude fewer
computational resources than is typically required for deep learning. Designing
neural networks for such a platform requires an intricate balance between
keeping high predictive performance (accuracy) while achieving low memory and
storage usage and inference latency. This is extremely challenging to achieve
manually, so in this work, we build a neural architecture search (NAS) system,
called $\mu$NAS, to automate the design of such small-yet-powerful MCU-level
networks. $\mu$NAS explicitly targets the three primary aspects of resource
scarcity of MCUs: the size of RAM, persistent storage and processor speed.
$\mu$NAS represents a significant advance in resource-efficient models,
especially for "mid-tier" MCUs with memory requirements ranging from 0.5 KB to
64 KB. We show that on a variety of image classification datasets $\mu$NAS is
able to (a) improve top-1 classification accuracy by up to 4.8%, or (b) reduce
memory footprint by 4--13x, or (c) reduce the number of multiply-accumulate
operations by at least 2x, compared to existing MCU specialist literature and
resource-efficient models.
