  Utilizing large pre-trained models for specific tasks has yielded impressive
results. However, fully fine-tuning these increasingly large models is becoming
prohibitively resource-intensive. This has led to a focus on more
parameter-efficient transfer learning, primarily within the same modality. But
this approach has limitations, particularly in video understanding where
suitable pre-trained models are less common. Addressing this, our study
introduces a novel cross-modality transfer learning approach from images to
videos, which we call parameter-efficient image-to-video transfer learning. We
present the Facial-Emotion Adapter (FE-Adapter), designed for efficient
fine-tuning in video tasks. This adapter allows pre-trained image models, which
traditionally lack temporal processing capabilities, to analyze dynamic video
content efficiently. Notably, it uses about 15 times fewer parameters than
previous methods, while improving accuracy. Our experiments in video emotion
recognition demonstrate that the FE-Adapter can match or even surpass existing
fine-tuning and video emotion models in both performance and efficiency. This
breakthrough highlights the potential for cross-modality approaches in
enhancing the capabilities of AI models, particularly in fields like video
emotion analysis where the demand for efficiency and accuracy is constantly
rising.
