  Self-training achieves enormous success in various semi-supervised and
weakly-supervised learning tasks. The method can be interpreted as a
teacher-student framework, where the teacher generates pseudo-labels, and the
student makes predictions. The two models are updated alternatingly. However,
such a straightforward alternating update rule leads to training instability.
This is because a small change in the teacher may result in a significant
change in the student. To address this issue, we propose DRIFT, short for
differentiable self-training, that treats teacher-student as a Stackelberg
game. In this game, a leader is always in a more advantageous position than a
follower. In self-training, the student contributes to the prediction
performance, and the teacher controls the training process by generating
pseudo-labels. Therefore, we treat the student as the leader and the teacher as
the follower. The leader procures its advantage by acknowledging the follower's
strategy, which involves differentiable pseudo-labels and differentiable sample
weights. Consequently, the leader-follower interaction can be effectively
captured via Stackelberg gradient, obtained by differentiating the follower's
strategy. Experimental results on semi- and weakly-supervised classification
and named entity recognition tasks show that our model outperforms existing
approaches by large margins.
