  Millions of images on the web enable us to explore images from social events
such as a family party, thus it is of interest to understand and model the
affect exhibited by a group of people in images. But analysis of the affect
expressed by multiple people is challenging due to varied indoor and outdoor
settings, and interactions taking place between various numbers of people. A
few existing works on Group-level Emotion Recognition (GER) have investigated
on face-level information. Due to the challenging environments, face may not
provide enough information to GER. Relatively few studies have investigated
multi-modal GER. Therefore, we propose a novel multi-modal approach based on a
new feature description for understanding emotional state of a group of people
in an image. In this paper, we firstly exploit three kinds of rich information
containing face, upperbody and scene in a group-level image. Furthermore, in
order to integrate multiple person's information in a group-level image, we
propose an information aggregation method to generate three features for face,
upperbody and scene, respectively. We fuse face, upperbody and scene
information for robustness of GER against the challenging environments.
Intensive experiments are performed on two challenging group-level emotion
databases to investigate the role of face, upperbody and scene as well as
multi-modal framework. Experimental results demonstrate that our framework
achieves very promising performance for GER.
