  Audio-Visual Segmentation (AVS) aims to extract the sounding object from a
video frame, which is represented by a pixel-wise segmentation mask for
application scenarios such as multi-modal video editing, augmented reality, and
intelligent robot systems. The pioneering work conducts this task through dense
feature-level audio-visual interaction, which ignores the dimension gap between
different modalities. More specifically, the audio clip could only provide a
Global semantic label in each sequence, but the video frame covers multiple
semantic objects across different Local regions, which leads to mislocalization
of the representationally similar but semantically different object. In this
paper, we propose a Cross-modal Cognitive Consensus guided Network (C3N) to
align the audio-visual semantics from the global dimension and progressively
inject them into the local regions via an attention mechanism. Firstly, a
Cross-modal Cognitive Consensus Inference Module (C3IM) is developed to extract
a unified-modal label by integrating audio/visual classification confidence and
similarities of modality-agnostic label embeddings. Then, we feed the
unified-modal label back to the visual backbone as the explicit semantic-level
guidance via a Cognitive Consensus guided Attention Module (CCAM), which
highlights the local features corresponding to the interested object. Extensive
experiments on the Single Sound Source Segmentation (S4) setting and Multiple
Sound Source Segmentation (MS3) setting of the AVSBench dataset demonstrate the
effectiveness of the proposed method, which achieves state-of-the-art
performance. Code is available at https://github.com/ZhaofengSHI/AVS-C3N.
