  In this paper, we propose a learning-based method to compose a video-story
from a group of video clips that describe an activity or experience. We learn
the coherence between video clips from real videos via the Recurrent Neural
Network (RNN) that jointly incorporates the spatial-temporal semantics and
motion dynamics to generate smooth and relevant compositions. We further
rearrange the results generated by the RNN to make the overall video-story
compatible with the storyline structure via a submodular ranking optimization
process. Experimental results on the video-story dataset show that the proposed
algorithm outperforms the state-of-the-art approach.
