  Different from other deep scalable architecture-based NAS approaches, Broad
Neural Architecture Search (BNAS) proposes a broad scalable architecture which
consists of convolution and enhancement blocks, dubbed Broad Convolutional
Neural Network (BCNN), as the search space for amazing efficiency improvement.
BCNN reuses the topologies of cells in the convolution block so that BNAS can
employ few cells for efficient search. Moreover, multi-scale feature fusion and
knowledge embedding are proposed to improve the performance of BCNN with
shallow topology. However, BNAS suffers some drawbacks: 1) insufficient
representation diversity for feature fusion and enhancement and 2) time
consumption of knowledge embedding design by human experts. This paper proposes
Stacked BNAS, whose search space is a developed broad scalable architecture
named Stacked BCNN, with better performance than BNAS. On the one hand, Stacked
BCNN treats mini BCNN as a basic block to preserve comprehensive representation
and deliver powerful feature extraction ability. For multi-scale feature
enhancement, each mini BCNN feeds the outputs of deep and broad cells to the
enhancement cell. For multi-scale feature fusion, each mini BCNN feeds the
outputs of deep, broad and enhancement cells to the output node. On the other
hand, Knowledge Embedding Search (KES) is proposed to learn appropriate
knowledge embeddings in a differentiable way. Moreover, the basic unit of KES
is an over-parameterized knowledge embedding module that consists of all
possible candidate knowledge embeddings. Experimental results show that 1)
Stacked BNAS obtains better performance than BNAS-v2 on both CIFAR-10 and
ImageNet, 2) the proposed KES algorithm contributes to reducing the parameters
of the learned architecture with satisfactory performance, and 3) Stacked BNAS
delivers a state-of-the-art efficiency of 0.02 GPU days.
