  Data augmentation is critical to the success of modern deep learning
techniques. In this paper, we propose Online Hyper-parameter Learning for
Auto-Augmentation (OHL-Auto-Aug), an economical solution that learns the
augmentation policy distribution along with network training. Unlike previous
methods on auto-augmentation that search augmentation strategies in an offline
manner, our method formulates the augmentation policy as a parameterized
probability distribution, thus allowing its parameters to be optimized jointly
with network parameters. Our proposed OHL-Auto-Aug eliminates the need of
re-training and dramatically reduces the cost of the overall search process,
while establishes significantly accuracy improvements over baseline models. On
both CIFAR-10 and ImageNet, our method achieves remarkable on search accuracy,
60x faster on CIFAR-10 and 24x faster on ImageNet, while maintaining
competitive accuracies.
