  For an interactive agent, such as task-oriented spoken dialog systems or
chatbots, measuring and adapting to Customer Satisfaction (CSAT) is critical in
order to understand user perception of an agent's behavior and increase user
engagement and retention. However, an agent often relies on explicit customer
feedback for measuring CSAT. Such explicit feedback may result in potential
distraction to users and it can be challenging to capture continuously changing
user's satisfaction. To address this challenge, we present a new approach to
automatically estimate CSAT using acoustic and lexical information in the Alexa
Prize Socialbot data. We first explore the relationship between CSAT and
sentiment scores at both the utterance and conversation level. We then
investigate static and temporal modeling methods that use estimated sentiment
scores as a mid-level representation. The results show that the sentiment
scores, particularly valence and satisfaction, are correlated with CSAT. We
also demonstrate that our proposed temporal modeling approach for estimating
CSAT achieves competitive performance, relative to static baselines as well as
human performance. This work provides insights into open domain social
conversations between real users and socialbots, and the use of both acoustic
and lexical information for understanding the relationship between CSAT and
sentiment scores.
