  We present a lightweight video motion retargeting approach TransMoMo that is
capable of transferring motion of a person in a source video realistically to
another video of a target person. Without using any paired data for
supervision, the proposed method can be trained in an unsupervised manner by
exploiting invariance properties of three orthogonal factors of variation
including motion, structure, and view-angle. Specifically, with loss functions
carefully derived based on invariance, we train an auto-encoder to disentangle
the latent representations of such factors given the source and target video
clips. This allows us to selectively transfer motion extracted from the source
video seamlessly to the target video in spite of structural and view-angle
disparities between the source and the target. The relaxed assumption of paired
data allows our method to be trained on a vast amount of videos needless of
manual annotation of source-target pairing, leading to improved robustness
against large structural variations and extreme motion in videos. We
demonstrate the effectiveness of our method over the state-of-the-art methods.
Code, model and data are publicly available on our project page
(https://yzhq97.github.io/transmomo).
