  We propose an end-to-end speaker-attributed automatic speech recognition
model that unifies speaker counting, speech recognition, and speaker
identification on monaural overlapped speech. Our model is built on serialized
output training (SOT) with attention-based encoder-decoder, a recently proposed
method for recognizing overlapped speech comprising an arbitrary number of
speakers. We extend SOT by introducing a speaker inventory as an auxiliary
input to produce speaker labels as well as multi-speaker transcriptions. All
model parameters are optimized by speaker-attributed maximum mutual information
criterion, which represents a joint probability for overlapped speech
recognition and speaker identification. Experiments on LibriSpeech corpus show
that our proposed method achieves significantly better speaker-attributed word
error rate than the baseline that separately performs overlapped speech
recognition and speaker identification.
