  As the number of video content has mushroomed in recent years, automatic
video summarization has come useful when we want to just peek at the content of
the video. However, there are two underlying limitations in generic video
summarization task. First, most previous approaches read in just visual
features as input, leaving other modality features behind. Second, existing
datasets for generic video summarization are relatively insufficient to train a
caption generator used for extracting text information from a video and to
train the multimodal feature extractors. To address these two problems, this
paper proposes the Multimodal Frame-Scoring Transformer (MFST), a framework
exploiting visual, text, and audio features and scoring a video with respect to
frames. Our MFST framework first extracts each modality features
(audio-visual-text) using pretrained encoders. Then, MFST trains the multimodal
frame-scoring transformer that uses multimodal representation based on
extracted features as inputs and predicts frame-level scores. Our extensive
experiments with previous models and ablation studies on TVSum and SumMe
datasets demonstrate the effectiveness and superiority of our proposed method
by a large margin in both F1 score and Rank-based evaluation.
