  This paper presents a novel system that enables intelligent robots to exhibit
realistic body gestures while communicating with humans. The proposed system
consists of a listening model and a speaking model used in corresponding
conversational phases. Both models are adapted from the sequence-to-sequence
(seq2seq) architecture to synthesize body gestures represented by the movements
of twelve upper-body keypoints. All the extracted 2D keypoints are firstly
3D-transformed, then rotated and normalized to discard irrelevant information.
Substantial videos of human conversations from Youtube are collected and
preprocessed to train the listening and speaking models separately, after which
the two models are evaluated using metrics of mean squared error (MSE) and
cosine similarity on the test dataset. The tuned system is implemented to drive
a virtual avatar as well as Pepper, a physical humanoid robot, to demonstrate
the improvement on conversational interaction abilities of our method in
practice.
