  Differentiable architecture search (DARTS) is an effective method for
data-driven neural network design based on solving a bilevel optimization
problem. Despite its success in many architecture search tasks, there are still
some concerns about the accuracy of first-order DARTS and the efficiency of the
second-order DARTS. In this paper, we formulate a single level alternative and
a relaxed architecture search (RARTS) method that utilizes the whole dataset in
architecture learning via both data and network splitting, without involving
mixed second derivatives of the corresponding loss functions like DARTS. In our
formulation of network splitting, two networks with different but related
weights cooperate in search of a shared architecture. The advantage of RARTS
over DARTS is justified by a convergence theorem and an analytically solvable
model. Moreover, RARTS outperforms DARTS and its variants in accuracy and
search efficiency, as shown in adequate experimental results. For the task of
searching topological architecture, i.e., the edges and the operations, RARTS
obtains a higher accuracy and 60\% reduction of computational cost than
second-order DARTS on CIFAR-10. RARTS continues to out-perform DARTS upon
transfer to ImageNet and is on par with recent variants of DARTS even though
our innovation is purely on the training algorithm without modifying search
space. For the task of searching width, i.e., the number of channels in
convolutional layers, RARTS also outperforms the traditional network pruning
benchmarks. Further experiments on the public architecture search benchmark
like NATS-Bench also support the preeminence of RARTS.
