  Minimal changes to neural architectures (e.g. changing a single
hyperparameter in a key layer), can lead to significant gains in predictive
performance in Convolutional Neural Networks (CNNs). In this work, we present a
new approach to receptive field analysis that can yield these types of
theoretical and empirical performance gains across twenty well-known CNN
architectures examined in our experiments. By further developing and
formalizing the analysis of receptive field expansion in convolutional neural
networks, we can predict unproductive layers in an automated manner before ever
training a model. This allows us to optimize the parameter-efficiency of a
given architecture at low cost. Our method is computationally simple and can be
done in an automated manner or even manually with minimal effort for most
common architectures. We demonstrate the effectiveness of this approach by
increasing parameter efficiency across past and current top-performing
CNN-architectures. Specifically, our approach is able to improve ImageNet1K
performance across a wide range of well-known, state-of-the-art (SOTA) model
classes, including: VGG Nets, MobileNetV1, MobileNetV3, NASNet A (mobile),
MnasNet, EfficientNet, and ConvNeXt - leading to a new SOTA result for each
model class.
