  In the domain of semi-supervised learning (SSL), the conventional approach
involves training a learner with a limited amount of labeled data alongside a
substantial volume of unlabeled data, both drawn from the same underlying
distribution. However, for deep learning models, this standard practice may not
yield optimal results. In this research, we propose an alternative perspective,
suggesting that distributions that are more readily separable could offer
superior benefits to the learner as compared to the original distribution. To
achieve this, we present PruneSSL, a practical technique for selectively
removing examples from the original unlabeled dataset to enhance its
separability. We present an empirical study, showing that although PruneSSL
reduces the quantity of available training data for the learner, it
significantly improves the performance of various competitive SSL algorithms,
thereby achieving state-of-the-art results across several image classification
tasks.
