  Empathy, as defined in behavioral sciences, expresses the ability of human
beings to recognize, understand and react to emotions, attitudes and beliefs of
others. The lack of an operational definition of empathy makes it difficult to
measure it. In this paper, we address two related problems in automatic
affective behavior analysis: the design of the annotation protocol and the
automatic recognition of empathy from spoken conversations. We propose and
evaluate an annotation scheme for empathy inspired by the modal model of
emotions. The annotation scheme was evaluated on a corpus of real-life, dyadic
spoken conversations. In the context of behavioral analysis, we designed an
automatic segmentation and classification system for empathy. Given the
different speech and language levels of representation where empathy may be
communicated, we investigated features derived from the lexical and acoustic
spaces. The feature development process was designed to support both the fusion
and automatic selection of relevant features from high dimensional space. The
automatic classification system was evaluated on call center conversations
where it showed significantly better performance than the baseline.
