  Speech transcription, emotion recognition, and language identification are
usually considered to be three different tasks. Each one requires a different
model with a different architecture and training process. We propose using a
recurrent neural network transducer (RNN-T)-based speech-to-text (STT) system
as a common component that can be used for emotion recognition and language
identification as well as for speech recognition. Our work extends the STT
system for emotion classification through minimal changes, and shows successful
results on the IEMOCAP and MELD datasets. In addition, we demonstrate that by
adding a lightweight component to the RNN-T module, it can also be used for
language identification. In our evaluations, this new classifier demonstrates
state-of-the-art accuracy for the NIST-LRE-07 dataset.
