  This paper proposes a self-supervised approach to learn universal facial
representations from videos, that can transfer across a variety of facial
analysis tasks such as Facial Attribute Recognition (FAR), Facial Expression
Recognition (FER), DeepFake Detection (DFD), and Lip Synchronization (LS). Our
proposed framework, named MARLIN, is a facial video masked autoencoder, that
learns highly robust and generic facial embeddings from abundantly available
non-annotated web crawled facial videos. As a challenging auxiliary task,
MARLIN reconstructs the spatio-temporal details of the face from the densely
masked facial regions which mainly include eyes, nose, mouth, lips, and skin to
capture local and global aspects that in turn help in encoding generic and
transferable features. Through a variety of experiments on diverse downstream
tasks, we demonstrate MARLIN to be an excellent facial video encoder as well as
feature extractor, that performs consistently well across a variety of
downstream tasks including FAR (1.13% gain over supervised benchmark), FER
(2.64% gain over unsupervised benchmark), DFD (1.86% gain over unsupervised
benchmark), LS (29.36% gain for Frechet Inception Distance), and even in low
data regime. Our code and models are available at
https://github.com/ControlNet/MARLIN .
