  We propose a weakly-supervised model for word-level mispronunciation
detection in non-native (L2) English speech. To train this model, phonetically
transcribed L2 speech is not required and we only need to mark mispronounced
words. The lack of phonetic transcriptions for L2 speech means that the model
has to learn only from a weak signal of word-level mispronunciations. Because
of that and due to the limited amount of mispronounced L2 speech, the model is
more likely to overfit. To limit this risk, we train it in a multi-task setup.
In the first task, we estimate the probabilities of word-level
mispronunciation. For the second task, we use a phoneme recognizer trained on
phonetically transcribed L1 speech that is easily accessible and can be
automatically annotated. Compared to state-of-the-art approaches, we improve
the accuracy of detecting word-level pronunciation errors in AUC metric by 30%
on the GUT Isle Corpus of L2 Polish speakers, and by 21.5% on the Isle Corpus
of L2 German and Italian speakers.
