  Appearance-based supervised methods with full-face image input have made
tremendous advances in recent gaze estimation tasks. However, intensive human
annotation requirement inhibits current methods from achieving industrial level
accuracy and robustness. Although current unsupervised pre-training frameworks
have achieved success in many image recognition tasks, due to the deep coupling
between facial and eye features, such frameworks are still deficient in
extracting useful gaze features from full-face. To alleviate above limitations,
this work proposes a novel unsupervised/self-supervised gaze pre-training
framework, which forces the full-face branch to learn a low dimensional gaze
embedding without gaze annotations, through collaborative feature contrast and
squeeze modules. In the heart of this framework is an alternating
eye-attended/unattended masking training scheme, which squeezes gaze-related
information from full-face branch into an eye-masked auto-encoder through an
injection bottleneck design that successfully encourages the model to pays more
attention to gaze direction rather than facial textures only, while still
adopting the eye self-reconstruction objective. In the same time, a novel
eye/gaze-related information contrastive loss has been designed to further
boost the learned representation by forcing the model to focus on eye-centered
regions. Extensive experimental results on several gaze benchmarks demonstrate
that the proposed scheme achieves superior performances over unsupervised
state-of-the-art.
