  Neural architecture search (NAS) algorithms save tremendous labor from human
experts. Recent advancements further reduce the computational overhead to an
affordable level. However, it is still cumbersome to deploy the NAS techniques
in real-world applications due to the fussy procedures and the supervised
learning paradigm. In this work, we propose the self-supervised and
weight-preserving neural architecture search (SSWP-NAS) as an extension of the
current NAS framework by allowing the self-supervision and retaining the
concomitant weights discovered during the search stage. As such, we simplify
the workflow of NAS to a one-stage and proxy-free procedure. Experiments show
that the architectures searched by the proposed framework achieve
state-of-the-art accuracy on CIFAR-10, CIFAR-100, and ImageNet datasets without
using manual labels. Moreover, we show that employing the concomitant weights
as initialization consistently outperforms the random initialization and the
two-stage weight pre-training method by a clear margin under semi-supervised
learning scenarios. Codes are publicly available at
https://github.com/LzVv123456/SSWP-NAS.
