  To fully understand the complexities of human emotion, the integration of
multiple physical features from different modalities can be advantageous.
Considering this, we present an analysis of 3D facial data, action units, and
physiological data as it relates to their impact on emotion recognition. We
analyze each modality independently, as well as the fusion of each for
recognizing human emotion. This analysis includes which features are most
important for specific emotions (e.g. happy). Our analysis indicates that both
3D facial landmarks and physiological data are encouraging for
expression/emotion recognition. On the other hand, while action units can
positively impact emotion recognition when fused with other modalities, the
results suggest it is difficult to detect emotion using them in a unimodal
fashion.
