  This paper studies learning on text-attributed graphs (TAGs), where each node
is associated with a text description. An ideal solution for such a problem
would be integrating both the text and graph structure information with large
language models and graph neural networks (GNNs). However, the problem becomes
very challenging when graphs are large due to the high computational complexity
brought by training large language models and GNNs together. In this paper, we
propose an efficient and effective solution to learning on large
text-attributed graphs by fusing graph structure and language learning with a
variational Expectation-Maximization (EM) framework, called GLEM. Instead of
simultaneously training large language models and GNNs on big graphs, GLEM
proposes to alternatively update the two modules in the E-step and M-step. Such
a procedure allows training the two modules separately while simultaneously
allowing the two modules to interact and mutually enhance each other. Extensive
experiments on multiple data sets demonstrate the efficiency and effectiveness
of the proposed approach.
