  Textual-edge Graphs (TEGs), characterized by rich text annotations on edges,
are increasingly significant in network science due to their ability to capture
rich contextual information among entities. Existing works have proposed
various edge-aware graph neural networks (GNNs) or let language models directly
make predictions. However, they often fall short of fully capturing the
contextualized semantics on edges and graph topology, respectively. This
inadequacy is particularly evident in link prediction tasks that require a
comprehensive understanding of graph topology and semantics between nodes. In
this paper, we present a novel framework - Link2Doc, designed especially for
link prediction on textual-edge graphs. Specifically, we propose to summarize
neighborhood information between node pairs as a human-written document to
preserve both semantic and topology information. A self-supervised learning
model is then utilized to enhance GNN's text-understanding ability from
language models. Empirical evaluations, including link prediction, edge
classification, parameter analysis, runtime comparison, and ablation studies,
on four real-world datasets demonstrate that Link2Doc achieves generally better
performance against existing edge-aware GNNs and pre-trained language models in
predicting links on TEGs.
