  Mixup is a popular data augmentation technique for training deep neural
networks where additional samples are generated by linearly interpolating pairs
of inputs and their labels. This technique is known to improve the
generalization performance in many learning paradigms and applications. In this
work, we first analyze Mixup and show that it implicitly regularizes infinitely
many directional derivatives of all orders. Based on this new insight, we
propose an improved version of Mixup, theoretically justified to deliver better
generalization performance than the vanilla Mixup. To demonstrate the
effectiveness of the proposed method, we conduct experiments across various
domains such as images, tabular data, speech, and graphs. Our results show that
the proposed method improves Mixup across multiple datasets using a variety of
architectures, for instance, exhibiting an improvement over Mixup by 0.8% in
ImageNet top-1 accuracy.
