  We address the weakly supervised video highlight detection problem for
learning to detect segments that are more attractive in training videos given
their video event label but without expensive supervision of manually
annotating highlight segments. While manually averting localizing highlight
segments, weakly supervised modeling is challenging, as a video in our daily
life could contain highlight segments with multiple event types, e.g., skiing
and surfing. In this work, we propose casting weakly supervised video highlight
detection modeling for a given specific event as a multiple instance ranking
network (MINI-Net) learning. We consider each video as a bag of segments, and
therefore, the proposed MINI-Net learns to enforce a higher highlight score for
a positive bag that contains highlight segments of a specific event than those
for negative bags that are irrelevant. In particular, we form a max-max ranking
loss to acquire a reliable relative comparison between the most likely positive
segment instance and the hardest negative segment instance. With this max-max
ranking loss, our MINI-Net effectively leverages all segment information to
acquire a more distinct video feature representation for localizing the
highlight segments of a specific event in a video. The extensive experimental
results on three challenging public benchmarks clearly validate the efficacy of
our multiple instance ranking approach for solving the problem.
