  Deep neural networks (DNNs) have demonstrated promising results in various
complex tasks. However, current DNNs encounter challenges with
over-parameterization, especially when there is limited training data
available. To enhance the generalization capability of DNNs, the Mixup
technique has gained popularity. Nevertheless, it still produces suboptimal
outcomes. Inspired by the successful Sharpness-Aware Minimization (SAM)
approach, which establishes a connection between the sharpness of the training
loss landscape and model generalization, we propose a new learning framework
called Generalized-Mixup, which combines the strengths of Mixup and SAM for
training DNN models. The theoretical analysis provided demonstrates how the
developed G-Mix framework enhances generalization. Additionally, to further
optimize DNN performance with the G-Mix framework, we introduce two novel
algorithms: Binary G-Mix and Decomposed G-Mix. These algorithms partition the
training data into two subsets based on the sharpness-sensitivity of each
example to address the issue of "manifold intrusion" in Mixup. Both theoretical
explanations and experimental results reveal that the proposed BG-Mix and
DG-Mix algorithms further enhance model generalization across multiple datasets
and models, achieving state-of-the-art performance.
