  Various normalization layers have been proposed to help the training of
neural networks. Group Normalization (GN) is one of the effective and
attractive studies that achieved significant performances in the visual
recognition task. Despite the great success achieved, GN still has several
issues that may negatively impact neural network training. In this paper, we
introduce an analysis framework and discuss the working principles of GN in
affecting the training process of the neural network. From experimental
results, we conclude the real cause of GN's inferior performance against Batch
normalization (BN): 1) \textbf{unstable training performance}, 2) \textbf{more
sensitive} to distortion, whether it comes from external noise or perturbations
introduced by the regularization. In addition, we found that GN can only help
the neural network training in some specific period, unlike BN, which helps the
network throughout the training. To solve these issues, we propose a new
normalization layer built on top of GN, by incorporating the advantages of BN.
Experimental results on the image classification task demonstrated that the
proposed normalization layer outperforms the official GN to improve recognition
accuracy regardless of the batch sizes and stabilize the network training.
