  Large Language Models (LLMs) have shown promising results on various language
and vision tasks. Recently, there has been growing interest in applying LLMs to
graph-based tasks, particularly on Text-Attributed Graphs (TAGs). However, most
studies have focused on node classification, while the use of LLMs for link
prediction (LP) remains understudied. In this work, we propose a new task on
LLMs, where the objective is to leverage LLMs to predict missing links between
nodes in a graph. This task evaluates an LLM's ability to reason over
structured data and infer new facts based on learned patterns. This new task
poses two key challenges: (1) How to effectively integrate pairwise structural
information into the LLMs, which is known to be crucial for LP performance, and
(2) how to solve the computational bottleneck when teaching LLMs to perform LP.
To address these challenges, we propose LinkGPT, the first end-to-end trained
LLM for LP tasks. To effectively enhance the LLM's ability to understand the
underlying structure, we design a two-stage instruction tuning approach where
the first stage fine-tunes the pairwise encoder, projector, and node projector,
and the second stage further fine-tunes the LLMs to predict links. To address
the efficiency challenges at inference time, we introduce a retrieval-reranking
scheme. Experiments show that LinkGPT can achieve state-of-the-art performance
on real-world graphs as well as superior generalization in zero-shot and
few-shot learning, surpassing existing benchmarks. At inference time, it can
achieve $10\times$ speedup while maintaining high LP accuracy.
