  Most recent semi-supervised deep learning (deep SSL) methods used a similar
paradigm: use network predictions to update pseudo-labels and use pseudo-labels
to update network parameters iteratively. However, they lack theoretical
support and cannot explain why predictions are good candidates for
pseudo-labels. In this paper, we propose a principled end-to-end framework
named deep decipher (D2) for SSL. Within the D2 framework, we prove that
pseudo-labels are related to network predictions by an exponential link
function, which gives a theoretical support for using predictions as
pseudo-labels. Furthermore, we demonstrate that updating pseudo-labels by
network predictions will make them uncertain. To mitigate this problem, we
propose a training strategy called repetitive reprediction (R2). Finally, the
proposed R2-D2 method is tested on the large-scale ImageNet dataset and
outperforms state-of-the-art methods by 5 percentage points.
