  Batch Normalization (BN) is an important preprocessing step to many deep
learning applications. Since it is a data-dependent process, for some
homogeneous datasets it is a redundant or even a performance-degrading process.
In this paper, we propose an early-stage feasibility assessment method for
estimating the benefits of applying BN on the given data batches. The proposed
method uses a novel threshold-based approach to classify the training data
batches into two sets according to their need for normalization. The need for
normalization is decided based on the feature heterogeneity of the considered
batch. The proposed approach is a pre-training processing, which implies no
training overhead. The evaluation results show that the proposed approach
achieves better performance mostly in small batch sizes than the traditional BN
using MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets. Additionally, the
network stability is increased by reducing the occurrence of internal variable
transformation.
