  Self-supervised facial representation has recently attracted increasing
attention due to its ability to perform face understanding without relying on
large-scale annotated datasets heavily. However, analytically, current
contrastive-based self-supervised learning (SSL) still performs
unsatisfactorily for learning facial representation. More specifically,
existing contrastive learning (CL) tends to learn pose-invariant features that
cannot depict the pose details of faces, compromising the learning performance.
To conquer the above limitation of CL, we propose a novel Pose-disentangled
Contrastive Learning (PCL) method for general self-supervised facial
representation. Our PCL first devises a pose-disentangled decoder (PDD) with a
delicately designed orthogonalizing regulation, which disentangles the
pose-related features from the face-aware features; therefore, pose-related and
other pose-unrelated facial information could be performed in individual
subnetworks and do not affect each other's training. Furthermore, we introduce
a pose-related contrastive learning scheme that learns pose-related information
based on data augmentation of the same image, which would deliver more
effective face-aware representation for various downstream tasks. We conducted
linear evaluation on four challenging downstream facial understanding tasks,
ie, facial expression recognition, face recognition, AU detection and head pose
estimation. Experimental results demonstrate that our method significantly
outperforms state-of-the-art SSL methods. Code is available at
https://github.com/DreamMr/PCL}{https://github.com/DreamMr/PCL
