  In this paper, alternating weak triphone/BPE alignment supervision is
proposed to improve end-to-end model training. Towards this end, triphone and
BPE alignments are extracted using a pre-existing hybrid ASR system. Then,
regularization effect is obtained by cross-entropy based intermediate auxiliary
losses computed on such alignments at a mid-layer representation of the encoder
for triphone alignments and at the encoder for BPE alignments. Weak supervision
is achieved through strong label smoothing with parameter of 0.5. Experimental
results on TED-LIUM 2 indicate that either triphone or BPE alignment based weak
supervision improves ASR performance over standard CTC auxiliary loss.
Moreover, their combination lowers the word error rate further. We also
investigate the alternation of the two auxiliary tasks during model training,
and additional performance gain is observed. Overall, the proposed techniques
result in over 10% relative error rate reduction over a CTC-regularized
baseline system.
