  Giving machines the ability to imagine possible new objects or scenes from
linguistic descriptions and produce their realistic renderings is arguably one
of the most challenging problems in computer vision. Recent advances in deep
generative models have led to new approaches that give promising results
towards this goal. In this paper, we introduce a new method called DiCoMoGAN
for manipulating videos with natural language, aiming to perform local and
semantic edits on a video clip to alter the appearances of an object of
interest. Our GAN architecture allows for better utilization of multiple
observations by disentangling content and motion to enable controllable
semantic edits. To this end, we introduce two tightly coupled networks: (i) a
representation network for constructing a concise understanding of motion
dynamics and temporally invariant content, and (ii) a translation network that
exploits the extracted latent content representation to actuate the
manipulation according to the target description. Our qualitative and
quantitative evaluations demonstrate that DiCoMoGAN significantly outperforms
existing frame-based methods, producing temporally coherent and semantically
more meaningful results.
