  Semi-supervised learning by self-training heavily relies on pseudo-label
selection (PLS). The selection often depends on the initial model fit on
labeled data. Early overfitting might thus be propagated to the final model by
selecting instances with overconfident but erroneous predictions, often
referred to as confirmation bias. This paper introduces BPLS, a Bayesian
framework for PLS that aims to mitigate this issue. At its core lies a
criterion for selecting instances to label: an analytical approximation of the
posterior predictive of pseudo-samples. We derive this selection criterion by
proving Bayes optimality of the posterior predictive of pseudo-samples. We
further overcome computational hurdles by approximating the criterion
analytically. Its relation to the marginal likelihood allows us to come up with
an approximation based on Laplace's method and the Gaussian integral. We
empirically assess BPLS for parametric generalized linear and non-parametric
generalized additive models on simulated and real-world data. When faced with
high-dimensional data prone to overfitting, BPLS outperforms traditional PLS
methods.
