  Predictor-based Neural Architecture Search (NAS) employs an architecture
performance predictor to improve the sample efficiency. However,
predictor-based NAS suffers from the severe ``cold-start'' problem, since a
large amount of architecture-performance data is required to get a working
predictor. In this paper, we focus on exploiting information in
cheaper-to-obtain performance estimations (i.e., low-fidelity information) to
mitigate the large data requirements of predictor training. Despite the
intuitiveness of this idea, we observe that using inappropriate low-fidelity
information even damages the prediction ability and different search spaces
have different preferences for low-fidelity information types. To solve the
problem and better fuse beneficial information provided by different types of
low-fidelity information, we propose a novel dynamic ensemble predictor
framework that comprises two steps. In the first step, we train different
sub-predictors on different types of available low-fidelity information to
extract beneficial knowledge as low-fidelity experts. In the second step, we
learn a gating network to dynamically output a set of weighting coefficients
conditioned on each input neural architecture, which will be used to combine
the predictions of different low-fidelity experts in a weighted sum. The
overall predictor is optimized on a small set of actual
architecture-performance data to fuse the knowledge from different low-fidelity
experts to make the final prediction. We conduct extensive experiments across
five search spaces with different architecture encoders under various
experimental settings. Our method can easily be incorporated into existing
predictor-based NAS frameworks to discover better architectures.
