  Despite significant advances in recent years, the existing Computer-Assisted
Pronunciation Training (CAPT) methods detect pronunciation errors with a
relatively low accuracy (precision of 60% at 40%-80% recall). This Ph.D. work
proposes novel deep learning methods for detecting pronunciation errors in
non-native (L2) English speech, outperforming the state-of-the-art method in
AUC metric (Area under the Curve) by 41%, i.e., from 0.528 to 0.749. One of the
problems with existing CAPT methods is the low availability of annotated
mispronounced speech needed for reliable training of pronunciation error
detection models. Therefore, the detection of pronunciation errors is
reformulated to the task of generating synthetic mispronounced speech.
Intuitively, if we could mimic mispronounced speech and produce any amount of
training data, detecting pronunciation errors would be more effective.
Furthermore, to eliminate the need to align canonical and recognized phonemes,
a novel end-to-end multi-task technique to directly detect pronunciation errors
was proposed. The pronunciation error detection models have been used at Amazon
to automatically detect pronunciation errors in synthetic speech to accelerate
the research into new speech synthesis methods. It was demonstrated that the
proposed deep learning methods are applicable in the tasks of detecting and
reconstructing dysarthric speech.
