  Discrete-Time Dynamic Graphs (DTDGs), which are prevalent in real-world
implementations and notable for their ease of data acquisition, have garnered
considerable attention from both academic researchers and industry
practitioners. The representation learning of DTDGs has been extensively
applied to model the dynamics of temporally changing entities and their
evolving connections. Currently, DTDG representation learning predominantly
relies on GNN+RNN architectures, which manifest the inherent limitations of
both Graph Neural Networks (GNNs) and Recurrent Neural Networks (RNNs). GNNs
suffer from the over-smoothing issue as the models architecture goes deeper,
while RNNs struggle to capture long-term dependencies effectively. GNN+RNN
architectures also grapple with scaling to large graph sizes and long
sequences. Additionally, these methods often compute node representations
separately and focus solely on individual node characteristics, thereby
overlooking the behavior intersections between the two nodes whose link is
being predicted, such as instances where the two nodes appear together in the
same context or share common neighbors.
  This paper introduces a novel representation learning method DTFormer for
DTDGs, pivoting from the traditional GNN+RNN framework to a Transformer-based
architecture. Our approach exploits the attention mechanism to concurrently
process topological information within the graph at each timestamp and temporal
dynamics of graphs along the timestamps, circumventing the aforementioned
fundamental weakness of both GNNs and RNNs. Moreover, we enhance the model's
expressive capability by incorporating the intersection relationships among
nodes and integrating a multi-patching module. Extensive experiments conducted
on six public dynamic graph benchmark datasets confirm our model's efficacy,
achieving the SOTA performance.
