  Historically, materials informatics has relied on human-designed descriptors
of materials structures. In recent years, graph neural networks (GNNs) have
been proposed for learning representations of crystal structures from data
end-to-end producing vectorial embeddings that are optimized for downstream
prediction tasks. However, a systematic scheme is lacking to analyze and
understand the limits of GNNs for capturing crystal structures. In this work,
we propose to use human-designed descriptors as a bank of human knowledge to
test whether black-box GNNs can capture the knowledge of crystal structures. We
find that current state-of-the-art GNNs cannot capture the periodicity of
crystal structures well, and we analyze the limitations of the GNN models that
result in this failure from three aspects: local expressive power, long-range
information, and readout function. We propose an initial solution, hybridizing
descriptors with GNNs, to improve the prediction of GNNs for materials
properties, especially phonon internal energy and heat capacity with 90% lower
errors, and we analyze the mechanisms for the improved prediction. All the
analysis can be extended easily to other deep representation learning models,
human-designed descriptors, and systems such as molecules and amorphous
materials.
