  Transformation invariances are present in many real-world problems. For
example, image classification is usually invariant to rotation and color
transformation: a rotated car in a different color is still identified as a
car. Data augmentation, which adds the transformed data into the training set
and trains a model on the augmented data, is one commonly used technique to
build these invariances into the learning process. However, it is unclear how
data augmentation performs theoretically and what the optimal algorithm is in
presence of transformation invariances. In this paper, we study PAC
learnability under transformation invariances in three settings according to
different levels of realizability: (i) A hypothesis fits the augmented data;
(ii) A hypothesis fits only the original data and the transformed data lying in
the support of the data distribution; (iii) Agnostic case. One interesting
observation is that distinguishing between the original data and the
transformed data is necessary to achieve optimal accuracy in setting (ii) and
(iii), which implies that any algorithm not differentiating between the
original and transformed data (including data augmentation) is not optimal.
Furthermore, this type of algorithms can even "harm" the accuracy. In setting
(i), although it is unnecessary to distinguish between the two data sets, data
augmentation still does not perform optimally. Due to such a difference, we
propose two combinatorial measures characterizing the optimal sample complexity
in setting (i) and (ii)(iii) and provide the optimal algorithms.
