  Weakly supervised temporal action localization (WS-TAL) is a challenging task
that aims to localize action instances in the given video with video-level
categorical supervision. Both appearance and motion features are used in
previous works, while they do not utilize them in a proper way but apply simple
concatenation or score-level fusion. In this work, we argue that the features
extracted from the pretrained extractor, e.g., I3D, are not the
WS-TALtask-specific features, thus the feature re-calibration is needed for
reducing the task-irrelevant information redundancy. Therefore, we propose a
cross-modal consensus network (CO2-Net) to tackle this problem. In CO2-Net, we
mainly introduce two identical proposed cross-modal consensus modules (CCM)
that design a cross-modal attention mechanism to filter out the task-irrelevant
information redundancy using the global information from the main modality and
the cross-modal local information of the auxiliary modality. Moreover, we treat
the attention weights derived from each CCMas the pseudo targets of the
attention weights derived from another CCM to maintain the consistency between
the predictions derived from two CCMs, forming a mutual learning manner.
Finally, we conduct extensive experiments on two common used temporal action
localization datasets, THUMOS14 and ActivityNet1.2, to verify our method and
achieve the state-of-the-art results. The experimental results show that our
proposed cross-modal consensus module can produce more representative features
for temporal action localization.
