  The combination of deep learning algorithm and materials science has made
significant progress in predicting novel materials and understanding various
behaviours of materials. Here, we introduced a new model called as the Crystal
Transformer Graph Neural Network (CTGNN), which combines the advantages of
Transformer model and graph neural networks to address the complexity of
structure-properties relation of material data. Compared to the
state-of-the-art models, CTGNN incorporates the graph network structure for
capturing local atomic interactions and the dual-Transformer structures to
model intra-crystal and inter-atomic relationships comprehensively. The
benchmark carried on by the proposed CTGNN indicates that CTGNN significantly
outperforms existing models like CGCNN and MEGNET in the prediction of
formation energy and bandgap properties. Our work highlights the potential of
CTGNN to enhance the performance of properties prediction and accelerates the
discovery of new materials, particularly for perovskite materials.
