  Most of multiple link prediction or graph generation techniques rely on the
attention mechanism or on Graph Neural Networks (GNNs), which consist in
leveraging node-level information exchanges in order to form proper link
predictions. Such node-level interactions do not process nodes as an ordered
sequence, which would imply some kind of natural ordering of the nodes: they
are said to be permutation invariant mechanisms. They are well suited for graph
problems, but struggle at providing a global orchestration of the predicted
links, which can result in a loss of performance. Some typical issues can be
the difficulty to ensure high-level properties such as global connectedness,
fixed diameter or to avoid information bottleneck effects such as oversmoothing
and oversquashing, which respectively consist in abundant smoothing in dense
areas leading to a loss of information and a tendency to exclude isolated nodes
from the message passing scheme, and often result in irrelevant, unbalanced
link predictions. To tackle this problem, we hereby present Cross-Attentive
Modulation (CAM) tokens, which introduce cross-attentive units used to
condition node and edge-level modulations in order to enable context-aware
computations that improve the global consistency of the prediction links. We
will implement it on a few permutation invariant architectures, and showcase
benchmarks that prove the merits of our work.
