  Human speech is often accompanied by hand and arm gestures. Given audio
speech input, we generate plausible gestures to go along with the sound.
Specifically, we perform cross-modal translation from "in-the-wild'' monologue
speech of a single speaker to their hand and arm motion. We train on unlabeled
videos for which we only have noisy pseudo ground truth from an automatic pose
detection system. Our proposed model significantly outperforms baseline methods
in a quantitative comparison. To support research toward obtaining a
computational understanding of the relationship between gesture and speech, we
release a large video dataset of person-specific gestures. The project website
with video, code and data can be found at
http://people.eecs.berkeley.edu/~shiry/speech2gesture .
