  Antiferromagnetic materials are exciting quantum materials with rich physics
and great potential for applications. It is highly demanded of the accurate and
efficient theoretical method for determining the critical transition
temperatures, N\'{e}el temperatures, of antiferromagnetic materials. The
powerful graph neural networks (GNN) that succeed in predicting material
properties lose their advantage in predicting magnetic properties due to the
small dataset of magnetic materials, while conventional machine learning models
heavily depend on the quality of material descriptors. We propose a new
strategy to extract high-level material representations by utilizing
self-supervised training of GNN on large-scale unlabeled datasets. According to
the dimensional reduction analysis, we find that the learned knowledge about
elements and magnetism transfers to the generated atomic vector
representations. Compared with popular manually constructed descriptors and
crystal graph convolutional neural networks, self-supervised material
representations can help us obtain a more accurate and efficient model for
N\'{e}el temperatures, and the trained model can successfully predict high
N\'{e}el temperature antiferromagnetic materials. Our self-supervised GNN may
serve as a universal pre-training framework for various material properties.
