  Driver action recognition, aiming to accurately identify drivers' behaviours,
is crucial for enhancing driver-vehicle interactions and ensuring driving
safety. Unlike general action recognition, drivers' environments are often
challenging, being gloomy and dark, and with the development of sensors,
various cameras such as IR and depth cameras have emerged for analyzing
drivers' behaviors. Therefore, in this paper, we propose a novel multimodal
fusion transformer, named MultiFuser, which identifies cross-modal
interrelations and interactions among multimodal car cabin videos and
adaptively integrates different modalities for improved representations.
Specifically, MultiFuser comprises layers of Bi-decomposed Modules to model
spatiotemporal features, with a modality synthesizer for multimodal features
integration. Each Bi-decomposed Module includes a Modal Expertise ViT block for
extracting modality-specific features and a Patch-wise Adaptive Fusion block
for efficient cross-modal fusion. Extensive experiments are conducted on
Drive&Act dataset and the results demonstrate the efficacy of our proposed
approach.
