  In this paper, we present a novel modeling method for single-channel
multi-talker overlapped automatic speech recognition (ASR) systems. Fully
neural network based end-to-end models have dramatically improved the
performance of multi-taker overlapped ASR tasks. One promising approach for
end-to-end modeling is autoregressive modeling with serialized output training
in which transcriptions of multiple speakers are recursively generated one
after another. This enables us to naturally capture relationships between
speakers. However, the conventional modeling method cannot explicitly take into
account the speaker attributes of individual utterances such as gender and age
information. In fact, the performance deteriorates when each speaker is the
same gender or is close in age. To address this problem, we propose unified
autoregressive modeling for joint end-to-end multi-talker overlapped ASR and
speaker attribute estimation. Our key idea is to handle gender and age
estimation tasks within the unified autoregressive modeling. In the proposed
method, transformer-based autoregressive model recursively generates not only
textual tokens but also attribute tokens of each speaker. This enables us to
effectively utilize speaker attributes for improving multi-talker overlapped
ASR. Experiments on Japanese multi-talker overlapped ASR tasks demonstrate the
effectiveness of the proposed method.
