  While semi-supervised learning (SSL) has received tremendous attentions in
many machine learning tasks due to its successful use of unlabeled data,
existing SSL algorithms use either all unlabeled examples or the unlabeled
examples with a fixed high-confidence prediction during the training progress.
However, it is possible that too many correct/wrong pseudo labeled examples are
eliminated/selected. In this work we develop a simple yet powerful framework,
whose key idea is to select a subset of training examples from the unlabeled
data when performing existing SSL methods so that only the unlabeled examples
with pseudo labels related to the labeled data will be used to train models.
The selection is performed at each updating iteration by only keeping the
examples whose losses are smaller than a given threshold that is dynamically
adjusted through the iteration. Our proposed approach, Dash, enjoys its
adaptivity in terms of unlabeled data selection and its theoretical guarantee.
Specifically, we theoretically establish the convergence rate of Dash from the
view of non-convex optimization. Finally, we empirically demonstrate the
effectiveness of the proposed method in comparison with state-of-the-art over
benchmarks.
