  Self-training is a simple yet effective method within semi-supervised
learning. The idea is to iteratively enhance training data by adding
pseudo-labeled data. Its generalization performance heavily depends on the
selection of these pseudo-labeled data (PLS). In this paper, we aim at
rendering PLS more robust towards the involved modeling assumptions. To this
end, we propose to select pseudo-labeled data that maximize a multi-objective
utility function. The latter is constructed to account for different sources of
uncertainty, three of which we discuss in more detail: model selection,
accumulation of errors and covariate shift. In the absence of second-order
information on such uncertainties, we furthermore consider the generic approach
of the generalized Bayesian alpha-cut updating rule for credal sets. As a
practical proof of concept, we spotlight the application of three of our robust
extensions on simulated and real-world data. Results suggest that in particular
robustness w.r.t. model choice can lead to substantial accuracy gains.
