  In this paper, we present a streaming end-to-end speech recognition model
based on Monotonic Chunkwise Attention (MoCha) jointly trained with enhancement
layers. Even though the MoCha attention enables streaming speech recognition
with recognition accuracy comparable to a full attention-based approach,
training this model is sensitive to various factors such as the difficulty of
training examples, hyper-parameters, and so on. Because of these issues, speech
recognition accuracy of a MoCha-based model for clean speech drops
significantly when a multi-style training approach is applied. Inspired by
Curriculum Learning [1], we introduce two training strategies: Gradual
Application of Enhanced Features (GAEF) and Gradual Reduction of Enhanced Loss
(GREL). With GAEF, the model is initially trained using clean features.
Subsequently, the portion of outputs from the enhancement layers gradually
increases. With GREL, the portion of the Mean Squared Error (MSE) loss for the
enhanced output gradually reduces as training proceeds. In experimental results
on the LibriSpeech corpus and noisy far-field test sets, the proposed model
with GAEF-GREL training strategies shows significantly better results than the
conventional multi-style training approach.
