  Crystal property prediction is a crucial aspect of developing novel
materials. However, there are two technical challenges to be addressed for
speeding up the investigation of crystals. First, labeling crystal properties
is intrinsically difficult due to the high cost and time involved in physical
simulations or lab experiments. Second, crystals adhere to a specific quantum
chemical principle known as periodic invariance, which is often not captured by
existing machine learning methods. To overcome these challenges, we propose the
crystal-specific pre-training framework for learning crystal representations
with self-supervision. The framework designs a mutex mask strategy for
enhancing representation learning so as to alleviate the limited labels
available for crystal property prediction. Moreover, we take into account the
specific periodic invariance in crystal structures by developing a periodic
invariance multi-graph module and periodic attribute learning within our
framework. This framework has been tested on eight different tasks. The
experimental results on these tasks show that the framework achieves promising
prediction performance and is able to outperform recent strong baselines.
