  Recently, there has been a growing interest in automating the process of
neural architecture design, and the Differentiable Architecture Search (DARTS)
method makes the process available within a few GPU days. However, the
performance of DARTS is often observed to collapse when the number of search
epochs becomes large. Meanwhile, lots of "{\em skip-connect}s" are found in the
selected architectures. In this paper, we claim that the cause of the collapse
is that there exists overfitting in the optimization of DARTS. Therefore, we
propose a simple and effective algorithm, named "DARTS+", to avoid the collapse
and improve the original DARTS, by "early stopping" the search procedure when
meeting a certain criterion. We also conduct comprehensive experiments on
benchmark datasets and different search spaces and show the effectiveness of
our DARTS+ algorithm, and DARTS+ achieves $2.32\%$ test error on CIFAR10,
$14.87\%$ on CIFAR100, and $23.7\%$ on ImageNet. We further remark that the
idea of "early stopping" is implicitly included in some existing DARTS variants
by manually setting a small number of search epochs, while we give an {\em
explicit} criterion for "early stopping".
