  Large language models (LLMs) are gaining increasing attention for their
capability to process graphs with rich text attributes, especially in a
zero-shot fashion. Recent studies demonstrate that LLMs obtain decent text
classification performance on common text-rich graph benchmarks, and the
performance can be improved by appending encoded structural information as
natural languages into prompts. We aim to understand why the incorporation of
structural information inherent in graph data can improve the prediction
performance of LLMs. First, we rule out the concern of data leakage by curating
a novel leakage-free dataset and conducting a comparative analysis alongside a
previously widely-used dataset. Second, as past work usually encodes the
ego-graph by describing the graph structure in natural language, we ask the
question: do LLMs understand the graph structure in accordance with the intent
of the prompt designers? Third, we investigate why LLMs can improve their
performance after incorporating structural information. Our exploration of
these questions reveals that (i) there is no substantial evidence that the
performance of LLMs is significantly attributed to data leakage; (ii) instead
of understanding prompts as graph structures as intended by the prompt
designers, LLMs tend to process prompts more as contextual paragraphs and (iii)
the most efficient elements of the local neighborhood included in the prompt
are phrases that are pertinent to the node label, rather than the graph
structure.
