  Graph Neural Networks (GNNs) have been widely applied to various fields due
to their powerful representations of graph-structured data. Despite the success
of GNNs, most existing GNNs are designed to learn node representations on the
fixed and homogeneous graphs. The limitations especially become problematic
when learning representations on a misspecified graph or a heterogeneous graph
that consists of various types of nodes and edges. To address this limitations,
we propose Graph Transformer Networks (GTNs) that are capable of generating new
graph structures, which preclude noisy connections and include useful
connections (e.g., meta-paths) for tasks, while learning effective node
representations on the new graphs in an end-to-end fashion. We further propose
enhanced version of GTNs, Fast Graph Transformer Networks (FastGTNs), that
improve scalability of graph transformations. Compared to GTNs, FastGTNs are
230x faster and use 100x less memory while allowing the identical graph
transformations as GTNs. In addition, we extend graph transformations to the
semantic proximity of nodes allowing non-local operations beyond meta-paths.
Extensive experiments on both homogeneous graphs and heterogeneous graphs show
that GTNs and FastGTNs with non-local operations achieve the state-of-the-art
performance for node classification tasks. The code is available:
https://github.com/seongjunyun/Graph_Transformer_Networks
