  Convolutional neural networks (CNNs) have become commonplace in addressing
major challenges in computer vision. Researchers are not only coming up with
new CNN architectures but are also researching different techniques to improve
the performance of existing architectures. However, there is a tendency to
over-emphasize performance improvement while neglecting certain important
variables such as simplicity, versatility, the fairness of comparisons, and
energy efficiency. Overlooking these variables in architectural design and
evaluation has led to research bias and a significantly negative environmental
impact. Furthermore, this can undermine the positive impact of research in
using deep learning models to tackle climate change. Here, we perform an
extensive and fair empirical study of a number of proposed techniques to gauge
the utility of each technique for segmentation and classification. Our findings
restate the importance of favoring simplicity over complexity in model design
(Occam's Razor). Furthermore, our results indicate that simple standardized
practices can lead to a significant reduction in environmental impact with
little drop in performance. We highlight that there is a need to rethink the
design and evaluation of CNNs to alleviate the issue of research bias and
carbon emissions.
