  Large datasets have been crucial to the success of deep learning models in
the recent years, which keep performing better as they are trained with more
labelled data. While there have been sustained efforts to make these models
more data-efficient, the potential benefit of understanding the data itself, is
largely untapped. Specifically, focusing on object recognition tasks, we wonder
if for common benchmark datasets we can do better than random subsets of the
data and find a subset that can generalize on par with the full dataset when
trained on. To our knowledge, this is the first result that can find notable
redundancies in CIFAR-10 and ImageNet datasets (at least 10%). Interestingly,
we observe semantic correlations between required and redundant images. We hope
that our findings can motivate further research into identifying additional
redundancies and exploiting them for more efficient training or
data-collection.
