  Data augmentation is a major component of many machine learning methods with
state-of-the-art performance. Common augmentation strategies work by drawing
random samples from a space of transformations. Unfortunately, such sampling
approaches are limited in expressivity, as they are unable to scale to rich
transformations that depend on numerous parameters due to the curse of
dimensionality. Adversarial examples can be considered as an alternative scheme
for data augmentation. By being trained on the most difficult modifications of
the inputs, the resulting models are then hopefully able to handle other,
presumably easier, modifications as well. The advantage of adversarial
augmentation is that it replaces sampling with the use of a single, calculated
perturbation that maximally increases the loss. The downside, however, is that
these raw adversarial perturbations appear rather unstructured; applying them
often does not produce a natural transformation, contrary to a desirable data
augmentation technique. To address this, we propose a method to generate
adversarial examples that maintain some desired natural structure. We first
construct a subspace that only contains perturbations with the desired
structure. We then project the raw adversarial gradient onto this space to
select a structured transformation that would maximally increase the loss when
applied. We demonstrate this approach through two types of image
transformations: photometric and geometric. Furthermore, we show that training
on such structured adversarial images improves generalization.
