  This paper addresses the challenge of out-of-distribution (OOD)
generalization in graph machine learning, a field rapidly advancing yet
grappling with the discrepancy between source and target data distributions.
Traditional graph learning algorithms, based on the assumption of uniform
distribution between training and test data, falter in real-world scenarios
where this assumption fails, resulting in suboptimal performance. A principal
factor contributing to this suboptimal performance is the inherent simplicity
bias of neural networks trained through Stochastic Gradient Descent (SGD),
which prefer simpler features over more complex yet equally or more predictive
ones. This bias leads to a reliance on spurious correlations, adversely
affecting OOD performance in various tasks such as image recognition, natural
language understanding, and graph classification. Current methodologies,
including subgraph-mixup and information bottleneck approaches, have achieved
partial success but struggle to overcome simplicity bias, often reinforcing
spurious correlations. To tackle this, we propose DIVE, training a collection
of models to focus on all label-predictive subgraphs by encouraging the models
to foster divergence on the subgraph mask, which circumvents the limitation of
a model solely focusing on the subgraph corresponding to simple structural
patterns. Specifically, we employs a regularizer to punish overlap in extracted
subgraphs across models, thereby encouraging different models to concentrate on
distinct structural patterns. Model selection for robust OOD performance is
achieved through validation accuracy. Tested across four datasets from GOOD
benchmark and one dataset from DrugOOD benchmark, our approach demonstrates
significant improvement over existing methods, effectively addressing the
simplicity bias and enhancing generalization in graph machine learning.
