  In many machine learning problems, large-scale datasets have become the
de-facto standard to train state-of-the-art deep networks at the price of heavy
computation load. In this paper, we focus on condensing large training sets
into significantly smaller synthetic sets which can be used to train deep
neural networks from scratch with minimum drop in performance. Inspired from
the recent training set synthesis methods, we propose Differentiable Siamese
Augmentation that enables effective use of data augmentation to synthesize more
informative synthetic images and thus achieves better performance when training
networks with augmentations. Experiments on multiple image classification
benchmarks demonstrate that the proposed method obtains substantial gains over
the state-of-the-art, 7% improvements on CIFAR10 and CIFAR100 datasets. We show
with only less than 1% data that our method achieves 99.6%, 94.9%, 88.5%, 71.5%
relative performance on MNIST, FashionMNIST, SVHN, CIFAR10 respectively. We
also explore the use of our method in continual learning and neural
architecture search, and show promising results.
