  Most recent semi-supervised deep learning (deep SSL) methods used a similar
paradigm: use network predictions to update pseudo-labels and use pseudo-labels
to update network parameters iteratively. However, they lack theoretical
support and cannot explain why predictions are good candidates for
pseudo-labels in the deep learning paradigm. In this paper, we propose a
principled end-to-end framework named deep decipher (D2) for SSL. Within the D2
framework, we prove that pseudo-labels are related to network predictions by an
exponential link function, which gives a theoretical support for using
predictions as pseudo-labels. Furthermore, we demonstrate that updating
pseudo-labels by network predictions will make them uncertain. To mitigate this
problem, we propose a training strategy called repetitive reprediction (R2).
Finally, the proposed R2-D2 method is tested on the large-scale ImageNet
dataset and outperforms state-of-the-art methods by 5 percentage points.
