  Dataset distillation is an emerging dataset reduction method, which condenses
large-scale datasets while maintaining task accuracy. Current methods have
integrated parameterization techniques to boost synthetic dataset performance
by shifting the optimization space from pixel to another informative feature
domain. However, they limit themselves to a fixed optimization space for
distillation, neglecting the diverse guidance across different informative
latent spaces. To overcome this limitation, we propose a novel parameterization
method dubbed Hierarchical Generative Latent Distillation (H-GLaD), to
systematically explore hierarchical layers within the generative adversarial
networks (GANs). This allows us to progressively span from the initial latent
space to the final pixel space. In addition, we introduce a novel
class-relevant feature distance metric to alleviate the computational burden
associated with synthetic dataset evaluation, bridging the gap between
synthetic and original datasets. Experimental results demonstrate that the
proposed H-GLaD achieves a significant improvement in both same-architecture
and cross-architecture performance with equivalent time consumption.
