  Human behavior refers to the way humans act and interact. Understanding human
behavior is a cornerstone of observational practice, especially in
psychotherapy. An important cue of behavior analysis is the dynamical changes
of emotions during the conversation. Domain experts integrate emotional
information in a highly nonlinear manner, thus, it is challenging to explicitly
quantify the relationship between emotions and behaviors. In this work, we
employ deep transfer learning to analyze their inferential capacity and
contextual importance. We first train a network to quantify emotions from
acoustic signals and then use information from the emotion recognition network
as features for behavior recognition. We treat this emotion-related information
as behavioral primitives and further train higher level layers towards behavior
quantification. Through our analysis, we find that emotion-related information
is an important cue for behavior recognition. Further, we investigate the
importance of emotional-context in the expression of behavior by constraining
(or not) the neural networks' contextual view of the data. This demonstrates
that the sequence of emotions is critical in behavior expression. To achieve
these frameworks we employ hybrid architectures of convolutional networks and
recurrent networks to extract emotion-related behavior primitives and
facilitate automatic behavior recognition from speech.
