  The automatic generation of stylized co-speech gestures has recently received
increasing attention. Previous systems typically allow style control via
predefined text labels or example motion clips, which are often not flexible
enough to convey user intent accurately. In this work, we present
GestureDiffuCLIP, a neural network framework for synthesizing realistic,
stylized co-speech gestures with flexible style control. We leverage the power
of the large-scale Contrastive-Language-Image-Pre-training (CLIP) model and
present a novel CLIP-guided mechanism that extracts efficient style
representations from multiple input modalities, such as a piece of text, an
example motion clip, or a video. Our system learns a latent diffusion model to
generate high-quality gestures and infuses the CLIP representations of style
into the generator via an adaptive instance normalization (AdaIN) layer. We
further devise a gesture-transcript alignment mechanism that ensures a
semantically correct gesture generation based on contrastive learning. Our
system can also be extended to allow fine-grained style control of individual
body parts. We demonstrate an extensive set of examples showing the flexibility
and generalizability of our model to a variety of style descriptions. In a user
study, we show that our system outperforms the state-of-the-art approaches
regarding human likeness, appropriateness, and style correctness.
