  Semi-supervised learning (SSL) provides an effective means of leveraging
unlabelled data to improve a model performance. Even though the domain has
received a considerable amount of attention in the past years, most methods
present the common drawback of lacking theoretical guarantees. Our starting
point is to notice that the estimate of the risk that most discriminative SSL
methods minimise is biased, even asymptotically. This bias impedes the use of
standard statistical learning theory and can hurt empirical performance. We
propose a simple way of removing the bias. Our debiasing approach is
straightforward to implement and applicable to most deep SSL methods. We
provide simple theoretical guarantees on the trustworthiness of these modified
methods, without having to rely on the strong assumptions on the data
distribution that SSL theory usually requires. In particular, we provide
generalisation error bounds for the proposed methods. We evaluate debiased
versions of different existing SSL methods, such as the Pseudo-label method and
Fixmatch, and show that debiasing can compete with classic deep SSL techniques
in various settings by providing better calibrated models. Additionally, we
provide a theoretical explanation of the intuition of the popular SSL methods.
