  Deep learning technology has developed unprecedentedly in the last decade and
has become the primary choice in many application domains. This progress is
mainly attributed to a systematic collaboration in which rapidly growing
computing resources encourage advanced algorithms to deal with massive data.
However, it has gradually become challenging to handle the unlimited growth of
data with limited computing power. To this end, diverse approaches are proposed
to improve data processing efficiency. Dataset distillation, a dataset
reduction method, addresses this problem by synthesizing a small typical
dataset from substantial data and has attracted much attention from the deep
learning community. Existing dataset distillation methods can be taxonomized
into meta-learning and data matching frameworks according to whether they
explicitly mimic the performance of target data. Although dataset distillation
has shown surprising performance in compressing datasets, there are still
several limitations such as distilling high-resolution data or data with
complex label spaces. This paper provides a holistic understanding of dataset
distillation from multiple aspects, including distillation frameworks and
algorithms, factorized dataset distillation, performance comparison, and
applications. Finally, we discuss challenges and promising directions to
further promote future studies on dataset distillation.
