  Speech Emotion Recognition (SER) is becoming a key role in global business
today to improve service efficiency, like call center services. Recent SERs
were based on a deep learning approach. However, the efficiency of deep
learning depends on the number of layers, i.e., the deeper layers, the higher
efficiency. On the other hand, the deeper layers are causes of a vanishing
gradient problem, a low learning rate, and high time-consuming. Therefore, this
paper proposed a redesign of existing local feature learning block (LFLB). The
new design is called a deep residual local feature learning block
(DeepResLFLB). DeepResLFLB consists of three cascade blocks: LFLB, residual
local feature learning block (ResLFLB), and multilayer perceptron (MLP). LFLB
is built for learning local correlations along with extracting hierarchical
correlations; DeepResLFLB can take advantage of repeatedly learning to explain
more detail in deeper layers using residual learning for solving vanishing
gradient and reducing overfitting; and MLP is adopted to find the relationship
of learning and discover probability for predicted speech emotions and gender
types. Based on two available published datasets: EMODB and RAVDESS, the
proposed DeepResLFLB can significantly improve performance when evaluated by
standard metrics: accuracy, precision, recall, and F1-score.
