  Recently, there emerged revived interests of designing automatic programs
(e.g., using genetic/evolutionary algorithms) to optimize the structure of
Convolutional Neural Networks (CNNs) for a specific task. The challenge in
designing such programs lies in how to balance between large search space of
the network structures and high computational costs. Existing works either
impose strong restrictions on the search space or use enormous computing
resources. In this paper, we study how to design a genetic programming approach
for optimizing the structure of a CNN for a given task under limited
computational resources yet without imposing strong restrictions on the search
space. To reduce the computational costs, we propose two general strategies
that are observed to be helpful: (i) aggressively selecting strongest
individuals for survival and reproduction, and killing weaker individuals at a
very early age; (ii) increasing mutation frequency to encourage diversity and
faster evolution. The combined strategy with additional optimization techniques
allows us to explore a large search space but with affordable computational
costs. Our results on standard benchmark datasets (MNIST, SVHN, CIFAR-10,
CIFAR-100) are competitive to similar approaches with significantly reduced
computational costs.
