  Studies on emotion recognition (ER) show that combining lexical and acoustic
information results in more robust and accurate models. The majority of the
studies focus on settings where both modalities are available in training and
evaluation. However, in practice, this is not always the case; getting ASR
output may represent a bottleneck in a deployment pipeline due to computational
complexity or privacy-related constraints. To address this challenge, we study
the problem of efficiently combining acoustic and lexical modalities during
training while still providing a deployable acoustic model that does not
require lexical inputs. We first experiment with multimodal models and two
attention mechanisms to assess the extent of the benefits that lexical
information can provide. Then, we frame the task as a multi-view learning
problem to induce semantic information from a multimodal model into our
acoustic-only network using a contrastive loss function. Our multimodal model
outperforms the previous state of the art on the USC-IEMOCAP dataset reported
on lexical and acoustic information. Additionally, our multi-view-trained
acoustic network significantly surpasses models that have been exclusively
trained with acoustic features.
