  Self-attentive neural syntactic parsers using contextualized word embeddings
(e.g. ELMo or BERT) currently produce state-of-the-art results in joint parsing
and disfluency detection in speech transcripts. Since the contextualized word
embeddings are pre-trained on a large amount of unlabeled data, using
additional unlabeled data to train a neural model might seem redundant.
However, we show that self-training - a semi-supervised technique for
incorporating unlabeled data - sets a new state-of-the-art for the
self-attentive parser on disfluency detection, demonstrating that self-training
provides benefits orthogonal to the pre-trained contextualized word
representations. We also show that ensembling self-trained parsers provides
further gains for disfluency detection.
