  Deep neural networks (DNNs) offer a means of addressing the challenging task
of clustering high-dimensional data. DNNs can extract useful features, and so
produce a lower dimensional representation, which is more amenable to
clustering techniques. As clustering is typically performed in a purely
unsupervised setting, where no training labels are available, the question then
arises as to how the DNN feature extractor can be trained. The most accurate
existing approaches combine the training of the DNN with the clustering
objective, so that information from the clustering process can be used to
update the DNN to produce better features for clustering. One problem with this
approach is that these ``pseudo-labels'' produced by the clustering algorithm
are noisy, and any errors that they contain will hurt the training of the DNN.
In this paper, we propose selective pseudo-label clustering, which uses only
the most confident pseudo-labels for training the~DNN. We formally prove the
performance gains under certain conditions. Applied to the task of image
clustering, the new approach achieves a state-of-the-art performance on three
popular image datasets. Code is available at
https://github.com/Lou1sM/clustering.
