  Due to the emergence of Generative Adversarial Networks, video synthesis has
witnessed exceptional breakthroughs. However, existing methods lack a proper
representation to explicitly control the dynamics in videos. Human pose, on the
other hand, can represent motion patterns intrinsically and interpretably, and
impose the geometric constraints regardless of appearance. In this paper, we
propose a pose guided method to synthesize human videos in a disentangled way:
plausible motion prediction and coherent appearance generation. In the first
stage, a Pose Sequence Generative Adversarial Network (PSGAN) learns in an
adversarial manner to yield pose sequences conditioned on the class label. In
the second stage, a Semantic Consistent Generative Adversarial Network (SCGAN)
generates video frames from the poses while preserving coherent appearances in
the input image. By enforcing semantic consistency between the generated and
ground-truth poses at a high feature level, our SCGAN is robust to noisy or
abnormal poses. Extensive experiments on both human action and human face
datasets manifest the superiority of the proposed method over other
state-of-the-arts.
