  Differentiable architecture search (DARTS) has significantly promoted the
development of NAS techniques because of its high search efficiency and
effectiveness but suffers from performance collapse. In this paper, we make
efforts to alleviate the performance collapse problem for DARTS from two
aspects. First, we investigate the expressive power of the supernet in DARTS
and then derive a new setup of DARTS paradigm with only training BatchNorm.
Second, we theoretically find that random features dilute the auxiliary
connection role of skip-connection in supernet optimization and enable search
algorithm focus on fairer operation selection, thereby solving the performance
collapse problem. We instantiate DARTS and PC-DARTS with random features to
build an improved version for each named RF-DARTS and RF-PCDARTS respectively.
Experimental results show that RF-DARTS obtains \textbf{94.36\%} test accuracy
on CIFAR-10 (which is the nearest optimal result in NAS-Bench-201), and
achieves the newest state-of-the-art top-1 test error of \textbf{24.0\%} on
ImageNet when transferring from CIFAR-10. Moreover, RF-DARTS performs robustly
across three datasets (CIFAR-10, CIFAR-100, and SVHN) and four search spaces
(S1-S4). Besides, RF-PCDARTS achieves even better results on ImageNet, that is,
\textbf{23.9\%} top-1 and \textbf{7.1\%} top-5 test error, surpassing
representative methods like single-path, training-free, and partial-channel
paradigms directly searched on ImageNet.
