  Data distillation is the problem of reducing the volume oftraining data while
keeping only the necessary information. With thispaper, we deeper explore the
new data distillation algorithm, previouslydesigned for image data. Our
experiments with tabular data show thatthe model trained on distilled samples
can outperform the model trainedon the original dataset. One of the problems of
the considered algorithmis that produced data has poor generalization on models
with differenthyperparameters. We show that using multiple architectures during
distillation can help overcome this problem.
