  Recently, a number of image-mixing-based augmentation techniques have been
introduced to improve the generalization of deep neural networks. In these
techniques, two or more randomly selected natural images are mixed together to
generate an augmented image. Such methods may not only omit important portions
of the input images but also introduce label ambiguities by mixing images
across labels resulting in misleading supervisory signals. To address these
limitations, we propose DiffuseMix, a novel data augmentation technique that
leverages a diffusion model to reshape training images, supervised by our
bespoke conditional prompts. First, concatenation of a partial natural image
and its generated counterpart is obtained which helps in avoiding the
generation of unrealistic images or label ambiguities. Then, to enhance
resilience against adversarial attacks and improves safety measures, a randomly
selected structural pattern from a set of fractal images is blended into the
concatenated image to form the final augmented image for training. Our
empirical results on seven different datasets reveal that DiffuseMix achieves
superior performance compared to existing state-of the-art methods on tasks
including general classification,fine-grained classification, fine-tuning, data
scarcity, and adversarial robustness. Augmented datasets and codes are
available here: https://diffusemix.github.io/
