  How to generate diverse, life-like, and unlimited long head/body sequences
without any driving source? We argue that this under-investigated research
problem is non-trivial at all, and has unique technical challenges behind it.
Without semantic constraints from the driving sources, using the standard
autoregressive model to generate infinitely long sequences would easily result
in 1) out-of-distribution (OOD) issue due to the accumulated error, 2)
insufficient diversity to produce natural and life-like motion sequences and 3)
undesired periodic patterns along the time. To tackle the above challenges, we
propose a systematic framework that marries the benefits of VQ-VAE and a novel
token-level control policy trained with reinforcement learning using carefully
designed reward functions. A high-level prior model can be easily injected on
top to generate unlimited long and diverse sequences. Although we focus on no
driving sources now, our framework can be generalized for controlled synthesis
with explicit driving sources. Through comprehensive evaluations, we conclude
that our proposed framework can address all the above-mentioned challenges and
outperform other strong baselines very significantly.
