  Modern neural networks are over-parameterized and thus rely on strong
regularization such as data augmentation and weight decay to reduce overfitting
and improve generalization. The dominant form of data augmentation applies
invariant transforms, where the learning target of a sample is invariant to the
transform applied to that sample. We draw inspiration from human visual
classification studies and propose generalizing augmentation with invariant
transforms to soft augmentation where the learning target softens non-linearly
as a function of the degree of the transform applied to the sample: e.g., more
aggressive image crop augmentations produce less confident learning targets. We
demonstrate that soft targets allow for more aggressive data augmentation,
offer more robust performance boosts, work with other augmentation policies,
and interestingly, produce better calibrated models (since they are trained to
be less confident on aggressively cropped/occluded examples). Combined with
existing aggressive augmentation strategies, soft target 1) doubles the top-1
accuracy boost across Cifar-10, Cifar-100, ImageNet-1K, and ImageNet-V2, 2)
improves model occlusion performance by up to $4\times$, and 3) halves the
expected calibration error (ECE). Finally, we show that soft augmentation
generalizes to self-supervised classification tasks. Code available at
https://github.com/youngleox/soft_augmentation
