  How romantic partners interact with each other during a conflict influences
how they feel at the end of the interaction and is predictive of whether the
partners stay together in the long term. Hence understanding the emotions of
each partner is important. Yet current approaches that are used include
self-reports which are burdensome and hence limit the frequency of this data
collection. Automatic emotion prediction could address this challenge. Insights
from psychology research indicate that partners' behaviors influence each
other's emotions in conflict interaction and hence, the behavior of both
partners could be considered to better predict each partner's emotion. However,
it is yet to be investigated how doing so compares to only using each partner's
own behavior in terms of emotion prediction performance. In this work, we used
BERT to extract linguistic features (i.e., what partners said) and openSMILE to
extract paralinguistic features (i.e., how they said it) from a data set of 368
German-speaking Swiss couples (N = 736 individuals) who were videotaped during
an 8-minutes conflict interaction in the laboratory. Based on those features,
we trained machine learning models to predict if partners feel positive or
negative after the conflict interaction. Our results show that including the
behavior of the other partner improves the prediction performance. Furthermore,
for men, considering how their female partners spoke is most important and for
women considering what their male partner said is most important in getting
better prediction performance. This work is a step towards automatically
recognizing each partners' emotion based on the behavior of both, which would
enable a better understanding of couples in research, therapy, and the real
world.
