  Cross-lingual speech emotion recognition (SER) is important for a wide range
of everyday applications. While recent SER research relies heavily on large
pretrained models for emotion training, existing studies often concentrate
solely on the final transformer layer of these models. However, given the
task-specific nature and hierarchical architecture of these models, each
transformer layer encapsulates different levels of information. Leveraging this
hierarchical structure, our study focuses on the information embedded across
different layers. Through an examination of layer feature similarity across
different languages, we propose a novel strategy called a layer-anchoring
mechanism to facilitate emotion transfer in cross-lingual SER tasks. Our
approach is evaluated using two distinct language affective corpora
(MSP-Podcast and BIIC-Podcast), achieving a best UAR performance of 60.21% on
the BIIC-podcast corpus. The analysis uncovers interesting insights into the
behavior of popular pretrained models.
