  Text generation from Abstract Meaning Representation (AMR) has substantially
benefited from the popularized Pretrained Language Models (PLMs). Myriad
approaches have linearized the input graph as a sequence of tokens to fit the
PLM tokenization requirements. Nevertheless, this transformation jeopardizes
the structural integrity of the graph and is therefore detrimental to its
resulting representation. To overcome this issue, Ribeiro et al. have recently
proposed StructAdapt, a structure-aware adapter which injects the input graph
connectivity within PLMs using Graph Neural Networks (GNNs). In this paper, we
investigate the influence of Relative Position Embeddings (RPE) on AMR-to-Text,
and, in parallel, we examine the robustness of StructAdapt. Through ablation
studies, graph attack and link prediction, we reveal that RPE might be
partially encoding input graphs. We suggest further research regarding the role
of RPE will provide valuable insights for Graph-to-Text generation.
