  Weakly supervised multimodal violence detection aims to learn a violence
detection model by leveraging multiple modalities such as RGB, optical flow,
and audio, while only video-level annotations are available. In the pursuit of
effective multimodal violence detection (MVD), information redundancy, modality
imbalance, and modality asynchrony are identified as three key challenges. In
this work, we propose a new weakly supervised MVD method that explicitly
addresses these challenges. Specifically, we introduce a multi-scale bottleneck
transformer (MSBT) based fusion module that employs a reduced number of
bottleneck tokens to gradually condense information and fuse each pair of
modalities and utilizes a bottleneck token-based weighting scheme to highlight
more important fused features. Furthermore, we propose a temporal consistency
contrast loss to semantically align pairwise fused features. Experiments on the
largest-scale XD-Violence dataset demonstrate that the proposed method achieves
state-of-the-art performance. Code is available at
https://github.com/shengyangsun/MSBT.
