  In this paper, we address the problem of effectively self-training neural
networks in a low-resource setting. Self-training is frequently used to
automatically increase the amount of training data. However, in a low-resource
scenario, it is less effective due to unreliable annotations created using
self-labeling of unlabeled data. We propose to combine self-training with noise
handling on the self-labeled data. Directly estimating noise on the combined
clean training set and self-labeled data can lead to corruption of the clean
data and hence, performs worse. Thus, we propose the Clean and Noisy Label
Neural Network which trains on clean and noisy self-labeled data simultaneously
by explicitly modelling clean and noisy labels separately. In our experiments
on Chunking and NER, this approach performs more robustly than the baselines.
Complementary to this explicit approach, noise can also be handled implicitly
with the help of an auxiliary learning task. To such a complementary approach,
our method is more beneficial than other baseline methods and together provides
the best performance overall.
