  Neural architecture search (NAS) has become increasingly popular in the deep
learning community recently, mainly because it can provide an opportunity to
allow interested users without rich expertise to benefit from the success of
deep neural networks (DNNs). However, NAS is still laborious and time-consuming
because a large number of performance estimations are required during the
search process of NAS, and training DNNs is computationally intensive. To solve
this major limitation of NAS, improving the computational efficiency is
essential in the design of NAS. However, a systematic overview of
computationally efficient NAS (CE-NAS) methods still lacks. To fill this gap,
we provide a comprehensive survey of the state-of-the-art on CE-NAS by
categorizing the existing work into proxy-based and surrogate-assisted NAS
methods, together with a thorough discussion of their design principles and a
quantitative comparison of their performances and computational complexities.
The remaining challenges and open research questions are also discussed, and
promising research topics in this emerging field are suggested.
