  Video summarization is among challenging tasks in computer vision, which aims
at identifying highlight frames or shots over a lengthy video input. In this
paper, we propose an novel attention-based framework for video summarization
with complex video data. Unlike previous works which only apply attention
mechanism on the correspondence between frames, our multi-concept video
self-attention (MC-VSA) model is presented to identify informative regions
across temporal and concept video features, which jointly exploit context
diversity over time and space for summarization purposes. Together with
consistency between video and summary enforced in our framework, our model can
be applied to both labeled and unlabeled data, making our method preferable to
real-world applications. Extensive and complete experiments on two benchmarks
demonstrate the effectiveness of our model both quantitatively and
qualitatively, and confirms its superiority over the stateof-the-arts.
