  We introduce a transformer-based GNN model, named UGformer, to learn graph
representations. In particular, we present two UGformer variants, wherein the
first variant (publicized in September 2019) is to leverage the transformer on
a set of sampled neighbors for each input node, while the second (publicized in
May 2021) is to leverage the transformer on all input nodes. Experimental
results demonstrate that the first UGformer variant achieves state-of-the-art
accuracies on benchmark datasets for graph classification in both inductive
setting and unsupervised transductive setting; and the second UGformer variant
obtains state-of-the-art accuracies for inductive text classification. The code
is available at: \url{https://github.com/daiquocnguyen/Graph-Transformer}.
