  Graph Neural Networks are notorious for its memory consumption. A recent
Transformer-based GNN called Graph Transformer is shown to obtain superior
performances when long range dependencies exist. However, combining graph data
and Transformer architecture led to a combinationally worse memory issue. We
propose a novel version of "edge regularization technique" that alleviates the
need for Positional Encoding and ultimately alleviate GT's out of memory issue.
We observe that it is not clear whether having an edge regularization on top of
positional encoding is helpful. However, it seems evident that applying our
edge regularization technique indeed stably improves GT's performance compared
to GT without Positional Encoding.
