  Dataset Distillation aims to compress a large dataset into a significantly
more compact, synthetic one without compromising the performance of the trained
models. To achieve this, existing methods use the agent model to extract
information from the target dataset and embed it into the distilled dataset.
Consequently, the quality of extracted and embedded information determines the
quality of the distilled dataset. In this work, we find that existing methods
introduce misaligned information in both information extraction and embedding
stages. To alleviate this, we propose Prioritize Alignment in Dataset
Distillation (PAD), which aligns information from the following two
perspectives. 1) We prune the target dataset according to the compressing ratio
to filter the information that can be extracted by the agent model. 2) We use
only deep layers of the agent model to perform the distillation to avoid
excessively introducing low-level information. This simple strategy effectively
filters out misaligned information and brings non-trivial improvement for
mainstream matching-based distillation algorithms. Furthermore, built on
trajectory matching, \textbf{PAD} achieves remarkable improvements on various
benchmarks, achieving state-of-the-art performance.
