  Audio-visual event (AVE) localization has attracted much attention in recent
years. Most existing methods are often limited to independently encoding and
classifying each video segment separated from the full video (which can be
regarded as the segment-level representations of events). However, they ignore
the semantic consistency of the event within the same full video (which can be
considered as the video-level representations of events). In contrast to
existing methods, we propose a novel video-level semantic consistency guidance
network for the AVE localization task. Specifically, we propose an event
semantic consistency modeling (ESCM) module to explore video-level semantic
information for semantic consistency modeling. It consists of two components: a
cross-modal event representation extractor (CERE) and an intra-modal semantic
consistency enhancer (ISCE). CERE is proposed to obtain the event semantic
information at the video level. Furthermore, ISCE takes video-level event
semantics as prior knowledge to guide the model to focus on the semantic
continuity of an event within each modality. Moreover, we propose a new
negative pair filter loss to encourage the network to filter out the irrelevant
segment pairs and a new smooth loss to further increase the gap between
different categories of events in the weakly-supervised setting. We perform
extensive experiments on the public AVE dataset and outperform the
state-of-the-art methods in both fully- and weakly-supervised settings, thus
verifying the effectiveness of our method.The code is available at
https://github.com/Bravo5542/VSCG.
