  Masked AutoEncoder (MAE) has revolutionized the field of self-supervised
learning with its simple yet effective masking and reconstruction strategies.
However, despite achieving state-of-the-art performance across various
downstream vision tasks, the underlying mechanisms that drive MAE's efficacy
are less well-explored compared to the canonical contrastive learning paradigm.
In this paper, we first propose a local perspective to explicitly extract a
local contrastive form from MAE's reconstructive objective at the patch level.
And then we introduce a new empirical framework, called Local Contrastive MAE
(LC-MAE), to analyze both reconstructive and contrastive aspects of MAE. LC-MAE
reveals that MAE learns invariance to random masking and ensures distribution
consistency between the learned token embeddings and the original images.
Furthermore, we dissect the contribution of the decoder and random masking to
MAE's success, revealing both the decoder's learning mechanism and the dual
role of random masking as data augmentation and effective receptive field
restriction. Our experimental analysis sheds light on the intricacies of MAE
and summarizes some useful design methodologies, which can inspire more
powerful visual self-supervised methods.
