  In this paper, we introduce a simple and novel framework for one-shot
audio-driven talking head generation. Unlike prior works that require
additional driving sources for controlled synthesis in a deterministic manner,
we instead probabilistically sample all the holistic lip-irrelevant facial
motions (i.e. pose, expression, blink, gaze, etc.) to semantically match the
input audio while still maintaining both the photo-realism of audio-lip
synchronization and the overall naturalness. This is achieved by our newly
proposed audio-to-visual diffusion prior trained on top of the mapping between
audio and disentangled non-lip facial representations. Thanks to the
probabilistic nature of the diffusion prior, one big advantage of our framework
is it can synthesize diverse facial motion sequences given the same audio clip,
which is quite user-friendly for many real applications. Through comprehensive
evaluations on public benchmarks, we conclude that (1) our diffusion prior
outperforms auto-regressive prior significantly on almost all the concerned
metrics; (2) our overall system is competitive with prior works in terms of
audio-lip synchronization but can effectively sample rich and natural-looking
lip-irrelevant facial motions while still semantically harmonized with the
audio input.
