  Recognizing emotions from speech is a daunting task due to the subtlety and
ambiguity of expressions. Traditional speech emotion recognition (SER) systems,
which typically rely on a singular, precise emotion label, struggle with this
complexity. Therefore, modeling the inherent ambiguity of emotions is an urgent
problem. In this paper, we propose an iterative prototype refinement framework
(IPR) for ambiguous SER. IPR comprises two interlinked components: contrastive
learning and class prototypes. The former provides an efficient way to obtain
high-quality representations of ambiguous samples. The latter are dynamically
updated based on ambiguous labels -- the similarity of the ambiguous data to
all prototypes. These refined embeddings yield precise pseudo labels, thus
reinforcing representation quality. Experimental evaluations conducted on the
IEMOCAP dataset validate the superior performance of IPR over state-of-the-art
methods, thus proving the effectiveness of our proposed method.
