  Dynamic neural networks (DyNNs) have become viable techniques to enable
intelligence on resource-constrained edge devices while maintaining
computational efficiency. In many cases, the implementation of DyNNs can be
sub-optimal due to its underlying backbone architecture being developed at the
design stage independent of both: (i) the dynamic computing features, e.g.
early exiting, and (ii) the resource efficiency features of the underlying
hardware, e.g., dynamic voltage and frequency scaling (DVFS). Addressing this,
we present HADAS, a novel Hardware-Aware Dynamic Neural Architecture Search
framework that realizes DyNN architectures whose backbone, early exiting
features, and DVFS settings have been jointly optimized to maximize performance
and resource efficiency. Our experiments using the CIFAR-100 dataset and a
diverse set of edge computing platforms have seen HADAS dynamic models achieve
up to 57% energy efficiency gains compared to the conventional dynamic ones
while maintaining the desired level of accuracy scores. Our code is available
at https://github.com/HalimaBouzidi/HADAS
