  While tokenized graph Transformers have demonstrated strong performance in
node classification tasks, their reliance on a limited subset of nodes with
high similarity scores for constructing token sequences overlooks valuable
information from other nodes, hindering their ability to fully harness graph
information for learning optimal node representations. To address this
limitation, we propose a novel graph Transformer called GCFormer. Unlike
previous approaches, GCFormer develops a hybrid token generator to create two
types of token sequences, positive and negative, to capture diverse graph
information. And a tailored Transformer-based backbone is adopted to learn
meaningful node representations from these generated token sequences.
Additionally, GCFormer introduces contrastive learning to extract valuable
information from both positive and negative token sequences, enhancing the
quality of learned node representations. Extensive experimental results across
various datasets, including homophily and heterophily graphs, demonstrate the
superiority of GCFormer in node classification, when compared to representative
graph neural networks (GNNs) and graph Transformers.
