  The lack of labeled data is a major obstacle in many music information
retrieval tasks such as melody extraction, where labeling is extremely
laborious or costly. Semi-supervised learning (SSL) provides a solution to
alleviate the issue by leveraging a large amount of unlabeled data. In this
paper, we propose an SSL method using teacher-student models for vocal melody
extraction. The teacher model is pre-trained with labeled data and guides the
student model to make identical predictions given unlabeled input in a
self-training setting. We examine three setups of teacher-student models with
different data augmentation schemes and loss functions. Also, considering the
scarcity of labeled data in the test phase, we artificially generate
large-scale testing data with pitch labels from unlabeled data using an
analysis-synthesis method. The results show that the SSL method significantly
increases the performance against supervised learning only and the improvement
depends on the teacher-student models, the size of unlabeled data, the number
of self-training iterations, and other training details. We also find that it
is essential to ensure that the unlabeled audio has vocal parts. Finally, we
show that the proposed SSL method enables a baseline convolutional recurrent
neural network model to achieve performance comparable to state-of-the-arts.
