  Doubles play an indispensable role in the movie industry. They take the place
of the actors in dangerous stunt scenes or scenes where the same actor plays
multiple characters. The double's face is later replaced with the actor's face
and expressions manually using expensive CGI technology, costing millions of
dollars and taking months to complete. An automated, inexpensive, and fast way
can be to use face-swapping techniques that aim to swap an identity from a
source face video (or an image) to a target face video. However, such methods
cannot preserve the source expressions of the actor important for the scene's
context. To tackle this challenge, we introduce video-to-video (V2V)
face-swapping, a novel task of face-swapping that can preserve (1) the identity
and expressions of the source (actor) face video and (2) the background and
pose of the target (double) video. We propose FaceOff, a V2V face-swapping
system that operates by learning a robust blending operation to merge two face
videos following the constraints above. It reduces the videos to a quantized
latent space and then blends them in the reduced space. FaceOff is trained in a
self-supervised manner and robustly tackles the non-trivial challenges of V2V
face-swapping. As shown in the experimental section, FaceOff significantly
outperforms alternate approaches qualitatively and quantitatively.
