  Identifying highlight moments of raw video materials is crucial for improving
the efficiency of editing videos that are pervasive on internet platforms.
However, the extensive work of manually labeling footage has created obstacles
to applying supervised methods to videos of unseen categories. The absence of
an audio modality that contains valuable cues for highlight detection in many
videos also makes it difficult to use multimodal strategies. In this paper, we
propose a novel model with cross-modal perception for unsupervised highlight
detection. The proposed model learns representations with visual-audio level
semantics from image-audio pair data via a self-reconstruction task. To achieve
unsupervised highlight detection, we investigate the latent representations of
the network and propose the representation activation sequence learning (RASL)
module with k-point contrastive learning to learn significant representation
activations. To connect the visual modality with the audio modality, we use the
symmetric contrastive learning (SCL) module to learn the paired visual and
audio representations. Furthermore, an auxiliary task of masked feature vector
sequence (FVS) reconstruction is simultaneously conducted during pretraining
for representation enhancement. During inference, the cross-modal pretrained
model can generate representations with paired visual-audio semantics given
only the visual modality. The RASL module is used to output the highlight
scores. The experimental results show that the proposed framework achieves
superior performance compared to other state-of-the-art approaches.
