  Designing accurate and efficient convolutional neural architectures for vast
amount of hardware is challenging because hardware designs are complex and
diverse. This paper addresses the hardware diversity challenge in Neural
Architecture Search (NAS). Unlike previous approaches that apply search
algorithms on a small, human-designed search space without considering hardware
diversity, we propose HURRICANE that explores the automatic hardware-aware
search over a much larger search space and a two-stage search algorithm, to
efficiently generate tailored models for different types of hardware. Extensive
experiments on ImageNet demonstrate that our algorithm outperforms
state-of-the-art hardware-aware NAS methods under the same latency constraint
on three types of hardware. Moreover, the discovered architectures achieve much
lower latency and higher accuracy than current state-of-the-art efficient
models. Remarkably, HURRICANE achieves a 76.67% top-1 accuracy on ImageNet with
a inference latency of only 16.5 ms for DSP, which is a 3.47% higher accuracy
and a 6.35x inference speedup than FBNet-iPhoneX, respectively. For VPU, we
achieve a 0.53% higher top-1 accuracy than Proxyless-mobile with a 1.49x
speedup. Even for well-studied mobile CPU, we achieve a 1.63% higher top-1
accuracy than FBNet-iPhoneX with a comparable inference latency. HURRICANE also
reduces the training time by 30.4% compared to SPOS.
