  Speech emotion recognition is the task of recognizing the speaker's emotional
state given a recording of their utterance. While most of the current
approaches focus on inferring emotion from isolated utterances, we argue that
this is not sufficient to achieve conversational emotion recognition (CER)
which deals with recognizing emotions in conversations. In this work, we
propose several approaches for CER by treating it as a sequence labeling task.
We investigated transformer architecture for CER and, compared it with
ResNet-34 and BiLSTM architectures in both contextual and context-less
scenarios using IEMOCAP corpus. Based on the inner workings of the
self-attention mechanism, we proposed DiverseCatAugment (DCA), an augmentation
scheme, which improved the transformer model performance by an absolute 3.3%
micro-f1 on conversations and 3.6% on isolated utterances. We further enhanced
the performance by introducing an interlocutor-aware transformer model where we
learn a dictionary of interlocutor index embeddings to exploit diarized
conversations.
