  Video processing and analysis have become an urgent task since a huge amount
of videos (e.g., Youtube, Hulu) are uploaded online every day. The extraction
of representative key frames from videos is very important in video processing
and analysis since it greatly reduces computing resources and time. Although
great progress has been made recently, large-scale video classification remains
an open problem, as the existing methods have not well balanced the performance
and efficiency simultaneously. To tackle this problem, this work presents an
unsupervised method to retrieve the key frames, which combines Convolutional
Neural Network (CNN) and Temporal Segment Density Peaks Clustering (TSDPC). The
proposed TSDPC is a generic and powerful framework and it has two advantages
compared with previous works, one is that it can calculate the number of key
frames automatically. The other is that it can preserve the temporal
information of the video. Thus it improves the efficiency of video
classification. Furthermore, a Long Short-Term Memory network (LSTM) is added
on the top of the CNN to further elevate the performance of classification.
Moreover, a weight fusion strategy of different input networks is presented to
boost the performance. By optimizing both video classification and key frame
extraction simultaneously, we achieve better classification performance and
higher efficiency. We evaluate our method on two popular datasets (i.e., HMDB51
and UCF101) and the experimental results consistently demonstrate that our
strategy achieves competitive performance and efficiency compared with the
state-of-the-art approaches.
