  Synthetic training data has gained prominence in numerous learning tasks and
scenarios, offering advantages such as dataset augmentation, generalization
evaluation, and privacy preservation. Despite these benefits, the efficiency of
synthetic data generated by current methodologies remains inferior when
training advanced deep models exclusively, limiting its practical utility. To
address this challenge, we analyze the principles underlying training data
synthesis for supervised learning and elucidate a principled theoretical
framework from the distribution-matching perspective that explicates the
mechanisms governing synthesis efficacy. Through extensive experiments, we
demonstrate the effectiveness of our synthetic data across diverse image
classification tasks, both as a replacement for and augmentation to real
datasets, while also benefits such as out-of-distribution generalization,
privacy preservation, and scalability. Specifically, we achieve 70.9% top1
classification accuracy on ImageNet1K when training solely with synthetic data
equivalent to 1 X the original real data size, which increases to 76.0% when
scaling up to 10 X synthetic data.
