  Recent studies have explored the use of pre-trained embeddings for speech
emotion recognition (SER), achieving comparable performance to conventional
methods that rely on low-level knowledge-inspired acoustic features. These
embeddings are often generated from models trained on large-scale speech
datasets using self-supervised or weakly-supervised learning objectives.
Despite the significant advancements made in SER through the use of pre-trained
embeddings, there is a limited understanding of the trustworthiness of these
methods, including privacy breaches, unfair performance, vulnerability to
adversarial attacks, and computational cost, all of which may hinder the
real-world deployment of these systems. In response, we introduce TrustSER, a
general framework designed to evaluate the trustworthiness of SER systems using
deep learning methods, with a focus on privacy, safety, fairness, and
sustainability, offering unique insights into future research in the field of
SER. Our code is publicly available under:
https://github.com/usc-sail/trust-ser.
