  Establishing evaluation schemes for spoken dialogue systems is important, but
it can also be challenging. While subjective evaluations are commonly used in
user experiments, objective evaluations are necessary for research comparison
and reproducibility. To address this issue, we propose a framework for
indirectly but objectively evaluating systems based on users' behaviors. In
this paper, to this end, we investigate the relationship between user behaviors
and subjective evaluation scores in social dialogue tasks: attentive listening,
job interview, and first-meeting conversation. The results reveal that in
dialogue tasks where user utterances are primary, such as attentive listening
and job interview, indicators like the number of utterances and words play a
significant role in evaluation. Observing disfluency also can indicate the
effectiveness of formal tasks, such as job interview. On the other hand, in
dialogue tasks with high interactivity, such as first-meeting conversation,
behaviors related to turn-taking, like average switch pause length, become more
important. These findings suggest that selecting appropriate user behaviors can
provide valuable insights for objective evaluation in each social dialogue
task.
