  We present DanceAnyWay, a generative learning method to synthesize
beat-guided dances of 3D human characters synchronized with music. Our method
learns to disentangle the dance movements at the beat frames from the dance
movements at all the remaining frames by operating at two hierarchical levels.
At the coarser "beat" level, it encodes the rhythm, pitch, and melody
information of the input music via dedicated feature representations only at
the beat frames. It leverages them to synthesize the beat poses of the target
dances using a sequence-to-sequence learning framework. At the finer
"repletion" level, our method encodes similar rhythm, pitch, and melody
information from all the frames of the input music via dedicated feature
representations. It generates the full dance sequences by combining the
synthesized beat and repletion poses and enforcing plausibility through an
adversarial learning framework. Our training paradigm also enforces
fine-grained diversity in the synthesized dances through a randomized temporal
contrastive loss, which ensures different segments of the dance sequences have
different movements and avoids motion freezing or collapsing to repetitive
movements. We evaluate the performance of our approach through extensive
experiments on the benchmark AIST++ dataset and observe improvements of about
7%-12% in motion quality metrics and 1.5%-4% in motion diversity metrics over
the current baselines, respectively. We also conducted a user study to evaluate
the visual quality of our synthesized dances. We note that, on average, the
samples generated by our method were about 9-48% more preferred by the
participants and had a 4-27% better five-point Likert-scale score over the best
available current baseline in terms of motion quality and synchronization. Our
source code and project page are available at
https://github.com/aneeshbhattacharya/DanceAnyWay.
