  The dominant paradigm for learning on graph-structured data is message
passing. Despite being a strong inductive bias, the local message passing
mechanism suffers from pathological issues such as over-smoothing,
over-squashing, and limited node-level expressivity. To address these
limitations we propose Bundle Neural Networks (BuNN), a new type of GNN that
operates via message diffusion over flat vector bundles - structures analogous
to connections on Riemannian manifolds that augment the graph by assigning to
each node a vector space and an orthogonal map. A BuNN layer evolves the
features according to a diffusion-type partial differential equation. When
discretized, BuNNs are a special case of Sheaf Neural Networks (SNNs), a
recently proposed MPNN capable of mitigating over-smoothing. The continuous
nature of message diffusion enables BuNNs to operate on larger scales of the
graph and, therefore, to mitigate over-squashing. Finally, we prove that BuNN
can approximate any feature transformation over nodes on any (potentially
infinite) family of graphs given injective positional encodings, resulting in
universal node-level expressivity. We support our theory via synthetic
experiments and showcase the strong empirical performance of BuNNs over a range
of real-world tasks, achieving state-of-the-art results on several standard
benchmarks in transductive and inductive settings.
