  Beyond improving trust and validating model fairness, xAI practices also have
the potential to recover valuable scientific insights in application domains
where little to no prior human intuition exists. To that end, we propose a
method to extract global concept explanations from the predictions of graph
neural networks to develop a deeper understanding of the tasks underlying
structure-property relationships. We identify concept explanations as dense
clusters in the self-explaining Megan models subgraph latent space. For each
concept, we optimize a representative prototype graph and optionally use GPT-4
to provide hypotheses about why each structure has a certain effect on the
prediction. We conduct computational experiments on synthetic and real-world
graph property prediction tasks. For the synthetic tasks we find that our
method correctly reproduces the structural rules by which they were created.
For real-world molecular property regression and classification tasks, we find
that our method rediscovers established rules of thumb. More specifically, our
results for molecular mutagenicity prediction indicate more fine-grained
resolution of structural details than existing explainability methods,
consistent with previous results from chemistry literature. Overall, our
results show promising capability to extract the underlying structure-property
relationships for complex graph property prediction tasks.
