  Understanding emotions is a fundamental aspect of human communication.
Integrating audio and video signals offers a more comprehensive understanding
of emotional states compared to traditional methods that rely on a single data
source, such as speech or facial expressions. Despite its potential, multimodal
emotion recognition faces significant challenges, particularly in
synchronization, feature extraction, and fusion of diverse data sources. To
address these issues, this paper introduces a novel transformer-based model
named Audio-Video Transformer Fusion with Cross Attention (AVT-CA). The AVT-CA
model employs a transformer fusion approach to effectively capture and
synchronize interlinked features from both audio and video inputs, thereby
resolving synchronization problems. Additionally, the Cross Attention mechanism
within AVT-CA selectively extracts and emphasizes critical features while
discarding irrelevant ones from both modalities, addressing feature extraction
and fusion challenges. Extensive experimental analysis conducted on the
CMU-MOSEI, RAVDESS and CREMA-D datasets demonstrates the efficacy of the
proposed model. The results underscore the importance of AVT-CA in developing
precise and reliable multimodal emotion recognition systems for practical
applications.
