The main goal of this thesis is to analyse film aesthetic experience evoked in spectators. In particular, we aim to detect aesthetic highlights in movies, as well as recognize induced emotions and aesthetic emotions elicited in spectators. The outcomes of the research on induced emotions, aesthetic emotions and aesthetic highlights can be used for emotional and aesthetic scene detection, emotional and aesthetic scene design, video summarization and prediction of affective and aesthetic content.
In this thesis, a background review on film aesthetic experience is provided. "Everyday" and aesthetic emotions are defined and a clear distinction between induced and perceived emotions of movie audiences is made. Several emotion representations and the characterization of emotion elicitation are discussed. The concept of interpersonal synchronization with regard to watching movies together is determined. An extensive literature review on aesthetic and affective content video analysis is also provided. Existing work on aesthetic and affect recognition, as well as highlight detection from video content and spectators’ reactions is described and discussed. The main limitations of the existing state of the art research are emphasized.
Currently available aesthetic and affective multimedia databases are described in details. The continuous LIRIS-ACCEDE database that was created to study film emotional experience in a movie theater is selected and extended to study film aesthetic experience. Protocols for collecting annotations of aesthetic highlights in movies, perceived emotions and aesthetic emotions felt by movie audiences are described. The statistical analysis of the annotations is carried out.
It is shown that aesthetic highlights in movies elicit a wide range of emotions. The amount of these emotions (a level of arousal and valence intensity) strongly depends on the aesthetic highlight category and on the movie genre. Also, methodology and results of aesthetic highlight detection based on the level of synchronization among spectators’ electrodermal activity (EDA) and acceleration (ACC) measurements are presented. The results suggest that the level of synchronization among spectators’ EDA and ACC signals is discriminative for aesthetic highlight detection in the context of watching movies together. In particular, pairwise synchronization measures are stable measures of synchronization and achieve the best performance of aesthetic highlight detection independently of movie genre and highlight categories.
The relationship between induced and perceived emotions of movie audiences is investigated. An inconsistency in induced and perceived emotion annotations is observed. In particular, it is found that induced and perceived emotions of movie audiences are not always positively correlated. Furthermore, it is observed that both perceived and induced emotions are characterized by aesthetic highlights. Finally, induced emotions are recognized from spectators’ EDA and ACC measurements as well as movie content. To this end we find that Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) models outperform Support Vector Regression (SVR) and Deep Belief Network (DBN) models because their ability to take into account temporal information and hierarchically combine knowledge-inspired affective cues with audiovisual movie content and movie audience responses.
It is shown that aesthetic highlights in movies evoke aesthetic emotions in spectators that are beyond "everyday" emotions. Aesthetic emotions that are felt by spectators are associated with the category of aesthetic highlights, as well as the movie genre. In fact, movie aesthetic emotions cannot be accurately described in the arousal valence space like "everyday" emotions. Four emotional dimensions that can accurately represent aesthetic emotions are found. Furthermore, the influence of personality on aesthetic emotions is assessed by noticing the differences in aesthetic scene ratings with regard to personality traits. Also, it is shown that aesthetic emotions can be predicted based on spectators’ reactions (EDA and ACC signals).