  Recently, zero-shot (or training-free) Neural Architecture Search (NAS)
approaches have been proposed to liberate NAS from the expensive training
process. The key idea behind zero-shot NAS approaches is to design proxies that
can predict the accuracy of some given networks without training the network
parameters. The proxies proposed so far are usually inspired by recent progress
in theoretical understanding of deep learning and have shown great potential on
several datasets and NAS benchmarks. This paper aims to comprehensively review
and compare the state-of-the-art (SOTA) zero-shot NAS approaches, with an
emphasis on their hardware awareness. To this end, we first review the
mainstream zero-shot proxies and discuss their theoretical underpinnings. We
then compare these zero-shot proxies through large-scale experiments and
demonstrate their effectiveness in both hardware-aware and hardware-oblivious
NAS scenarios. Finally, we point out several promising ideas to design better
proxies. Our source code and the list of related papers are available on
https://github.com/SLDGroup/survey-zero-shot-nas.
