  The manual annotation for large-scale point clouds is still tedious and
unavailable for many harsh real-world tasks. Self-supervised learning, which is
used on raw and unlabeled data to pre-train deep neural networks, is a
promising approach to address this issue. Existing works usually take the
common aid from auto-encoders to establish the self-supervision by the
self-reconstruction schema. However, the previous auto-encoders merely focus on
the global shapes and do not distinguish the local and global geometric
features apart. To address this problem, we present a novel and efficient
self-supervised point cloud representation learning framework, named 3D
Occlusion Auto-Encoder (3D-OAE), to facilitate the detailed supervision
inherited in local regions and global shapes. We propose to randomly occlude
some local patches of point clouds and establish the supervision via inpainting
the occluded patches using the remaining ones. Specifically, we design an
asymmetrical encoder-decoder architecture based on standard Transformer, where
the encoder operates only on the visible subset of patches to learn local
patterns, and a lightweight decoder is designed to leverage these visible
patterns to infer the missing geometries via self-attention. We find that
occluding a very high proportion of the input point cloud (e.g. 75%) will still
yield a nontrivial self-supervisory performance, which enables us to achieve
3-4 times faster during training but also improve accuracy. Experimental
results show that our approach outperforms the state-of-the-art on a diverse
range of downstream discriminative and generative tasks.
