  In this paper, we present a novel method for dynamically expanding
Convolutional Neural Networks (CNNs) during training, aimed at meeting the
increasing demand for efficient and sustainable deep learning models. Our
approach, drawing from the seminal work on Self-Expanding Neural Networks
(SENN), employs a natural expansion score as an expansion criteria to address
the common issue of over-parameterization in deep convolutional neural
networks, thereby ensuring that the model's complexity is finely tuned to the
task's specific needs. A significant benefit of this method is its eco-friendly
nature, as it obviates the necessity of training multiple models of different
sizes. We employ a strategy where a single model is dynamically expanded,
facilitating the extraction of checkpoints at various complexity levels,
effectively reducing computational resource use and energy consumption while
also expediting the development cycle by offering diverse model complexities
from a single training session. We evaluate our method on the CIFAR-10 dataset
and our experimental results validate this approach, demonstrating that
dynamically adding layers not only maintains but also improves CNN performance,
underscoring the effectiveness of our expansion criteria. This approach marks a
considerable advancement in developing adaptive, scalable, and environmentally
considerate neural network architectures, addressing key challenges in the
field of deep learning.
