  Recent advances in diffusion models have showcased promising results in the
text-to-video (T2V) synthesis task. However, as these T2V models solely employ
text as the guidance, they tend to struggle in modeling detailed temporal
dynamics. In this paper, we introduce a novel T2V framework that additionally
employ audio signals to control the temporal dynamics, empowering an
off-the-shelf T2I diffusion to generate audio-aligned videos. We propose
audio-based regional editing and signal smoothing to strike a good balance
between the two contradicting desiderata of video synthesis, i.e., temporal
flexibility and coherence. We empirically demonstrate the effectiveness of our
method through experiments, and further present practical applications for
contents creation.
