  Differentiable neural architecture search (DARTS) has gained much success in
discovering flexible and diverse cell types. To reduce the evaluation gap, the
supernet is expected to have identical layers with the target network. However,
even for this consistent search, the searched cells often suffer from poor
performance, especially for the supernet with fewer layers, as current DARTS
methods are prone to wide and shallow cells, and this topology collapse induces
sub-optimal searched cells. In this paper, we alleviate this issue by endowing
the cells with explicit stretchability, so the search can be directly
implemented on our stretchable cells for both operation type and topology
simultaneously. Concretely, we introduce a set of topological variables and a
combinatorial probabilistic distribution to explicitly model the target
topology. With more diverse and complex topologies, our method adapts well for
various layer numbers. Extensive experiments on CIFAR-10 and ImageNet show that
our stretchable cells obtain better performance with fewer layers and
parameters. For example, our method can improve DARTS by 0.28\% accuracy on
CIFAR-10 dataset with 45\% parameters reduced or 2.9\% with similar FLOPs on
ImageNet dataset.
