  Differentiable ARchiTecture Search (DARTS) is one of the most trending Neural
Architecture Search (NAS) methods. It drastically reduces search cost by
resorting to weight-sharing. However, it also dramatically reduces the search
space, thus excluding potential promising architectures. In this article, we
propose D-DARTS, a solution that addresses this problem by nesting neural
networks at the cell level instead of using weight-sharing to produce more
diversified and specialized architectures. Moreover, we introduce a novel
algorithm that can derive deeper architectures from a few trained cells,
increasing performance and saving computation time. In addition, we also
present an alternative search space (DARTOpti) in which we optimize existing
handcrafted architectures (e.g., ResNet) rather than starting from scratch.
This approach is accompanied by a novel metric that measures the distance
between architectures inside our custom search space. Our solution reaches
competitive performance on multiple computer vision tasks. Code and pretrained
models can be accessed at https://github.com/aheuillet/D-DARTS.
