  The affect embedded in video data conveys high-level semantic information
about the content and has direct impact on the understanding and perception of
reviewers, as well as their emotional responses. Affective Video Content
Analysis (AVCA) attempts to generate a direct mapping between video content and
the corresponding affective states such as arousal and valence dimensions. Most
existing studies establish the mapping for each dimension separately using
knowledge-based rules or traditional classifiers such as Support Vector Machine
(SVM). The inherent correlations between affective dimensions have largely been
unexploited, which are anticipated to include important information for
accurate prediction of affective dimensions. To address this issue, this paper
presents an approach to predict arousal and valance dimensions synchronously
using the Long Short Term Memory (LSTM) network. The approach extracts a set of
low-level audio and visual features from video data and projects them
synchronously into pairs of arousal and valence values using the LSTM network
which automatically incorporates the correlations between arousal and valance
dimensions. We evaluate the performance of the proposed approach on a dataset
comprising video clips segmented from real-world resources such as film, drama,
and news, and demonstrate its superior performance over the traditional SVM
based method. The results provide one of the earliest preliminary evidence to
the benefit of considering correlations between affective dimensions towards
accurate AVCA.
