  Architecture search optimizes the structure of a neural network for some task
instead of relying on manual authoring. However, it is slow, as each potential
architecture is typically trained from scratch. In this paper we present an
approach called Conceptual Expansion Neural Architecture Search (CENAS) that
combines a sample-efficient, computational creativity-inspired transfer
learning approach with neural architecture search. This approach finds models
faster than naive architecture search via transferring existing weights to
approximate the parameters of the new model. It outperforms standard transfer
learning by allowing for the addition of features instead of only modifying
existing features. We demonstrate that our approach outperforms standard neural
architecture search and transfer learning methods in terms of efficiency,
performance, and parameter counts on a variety of transfer learning tasks.
