  In recent years, speech emotion recognition (SER) has been used in wide
ranging applications, from healthcare to the commercial sector. In addition to
signal processing approaches, methods for SER now also use deep learning
techniques. However, generalizing over languages, corpora and recording
conditions is still an open challenge in the field. Furthermore, due to the
black-box nature of deep learning algorithms, a newer challenge is the lack of
interpretation and transparency in the models and the decision making process.
This is critical when the SER systems are deployed in applications that
influence human lives. In this work we address this gap by providing an
in-depth analysis of the decision making process of the proposed SER system.
Towards that end, we present low-complexity SER based on undercomplete- and
denoising- autoencoders that achieve an average classification accuracy of over
55\% for four-class emotion classification. Following this, we investigate the
clustering of emotions in the latent space to understand the influence of the
corpora on the model behavior and to obtain a physical interpretation of the
latent embedding. Lastly, we explore the role of each input feature towards the
performance of the SER.
