  In this paper, an end-to-end neural embedding system based on triplet loss
and residual learning has been proposed for speech emotion recognition. The
proposed system learns the embeddings from the emotional information of the
speech utterances. The learned embeddings are used to recognize the emotions
portrayed by given speech samples of various lengths. The proposed system
implements Residual Neural Network architecture. It is trained using softmax
pre-training and triplet loss function. The weights between the fully connected
and embedding layers of the trained network are used to calculate the embedding
values. The embedding representations of various emotions are mapped onto a
hyperplane, and the angles among them are computed using the cosine similarity.
These angles are utilized to classify a new speech sample into its appropriate
emotion class. The proposed system has demonstrated 91.67% and 64.44% accuracy
while recognizing emotions for RAVDESS and IEMOCAP dataset, respectively.
