  Mixup is a data augmentation technique that relies on training using random
convex combinations of data points and their labels. In recent years, Mixup has
become a standard primitive used in the training of state-of-the-art image
classification models due to its demonstrated benefits over empirical risk
minimization with regards to generalization and robustness. In this work, we
try to explain some of this success from a feature learning perspective. We
focus our attention on classification problems in which each class may have
multiple associated features (or views) that can be used to predict the class
correctly. Our main theoretical results demonstrate that, for a non-trivial
class of data distributions with two features per class, training a 2-layer
convolutional network using empirical risk minimization can lead to learning
only one feature for almost all classes while training with a specific
instantiation of Mixup succeeds in learning both features for every class. We
also show empirically that these theoretical insights extend to the practical
settings of image benchmarks modified to have multiple features.
