  Real-world data generation often involves certain geometries (e.g., graphs)
that induce instance-level interdependence. This characteristic makes the
generalization of learning models more difficult due to the intricate
interdependent patterns that impact data-generative distributions and can vary
from training to testing. In this work, we propose a geometric diffusion model
with learnable divergence fields for the challenging generalization problem
with interdependent data. We generalize the diffusion equation with stochastic
diffusivity at each time step, which aims to capture the multi-faceted
information flows among interdependent data. Furthermore, we derive a new
learning objective through causal inference, which can guide the model to learn
generalizable patterns of interdependence that are insensitive across domains.
Regarding practical implementation, we introduce three model instantiations
that can be considered as the generalized versions of GCN, GAT, and
Transformers, respectively, which possess advanced robustness against
distribution shifts. We demonstrate their promising efficacy for
out-of-distribution generalization on diverse real-world datasets.
