  Embodied conversational agents (ECA) are often designed to produce nonverbal
behavior to complement or enhance their verbal communication. One such form of
nonverbal behavior is co-speech gesturing, which involves movements that the
agent makes with its arms and hands that are paired with verbal communication.
Co-speech gestures for ECAs can be created using different generation methods,
divided into rule-based and data-driven processes, with the latter gaining
traction because of the increasing interest from the applied machine learning
community. However, reports on gesture generation methods use a variety of
evaluation measures, which hinders comparison. To address this, we present a
systematic review on co-speech gesture generation methods for iconic,
metaphoric, deictic, and beat gestures, including reported evaluation methods.
We review 22 studies that have an ECA with a human-like upper body that uses
co-speech gesturing in social human-agent interaction. This includes studies
that use human participants to evaluate performance. We found most studies use
a within-subject design and rely on a form of subjective evaluation, but
without a systematic approach. We argue that the field requires more rigorous
and uniform tools for co-speech gesture evaluation, and formulate
recommendations for empirical evaluation, including standardized phrases and
example scenarios to help systematically test generative models across studies.
Furthermore, we also propose a checklist that can be used to report relevant
information for the evaluation of generative models, as well as to evaluate
co-speech gesture use.
