  Neural architecture search (NAS), which automatically designs the
architectures of deep neural networks, has achieved breakthrough success over
many applications in the past few years. Among different classes of NAS
methods, evolutionary computation based NAS (ENAS) methods have recently gained
much attention. Unfortunately, the issues of fair comparisons and efficient
evaluations have hindered the development of ENAS. The current benchmark
architecture datasets designed for fair comparisons only provide the datasets,
not the ENAS algorithms or the platform to run the algorithms. The existing
efficient evaluation methods are either not suitable for the population-based
ENAS algorithm or are too complex to use. This paper develops a platform named
BenchENAS to address these issues. BenchENAS aims to achieve fair comparisons
by running different algorithms in the same environment and with the same
settings. To achieve efficient evaluation in a common lab environment,
BenchENAS designs a parallel component and a cache component with high
maintainability. Furthermore, BenchENAS is easy to install and highly
configurable and modular, which brings benefits in good usability and easy
extensibility. The paper conducts efficient comparison experiments on eight
ENAS algorithms with high GPU utilization on this platform. The experiments
validate that the fair comparison issue does exist, and BenchENAS can alleviate
this issue. A website has been built to promote BenchENAS at
https://benchenas.com, where interested researchers can obtain the source code
and document of BenchENAS for free.
