  Large-scale pre-trained language models (PLMs) have advanced Graph-to-Text
(G2T) generation by processing the linearised version of a graph. However, the
linearisation is known to ignore the structural information. Additionally, PLMs
are typically pre-trained on free text which introduces domain mismatch between
pre-training and downstream G2T generation tasks. To address these
shortcomings, we propose graph masking pre-training strategies that neither
require supervision signals nor adjust the architecture of the underlying
pre-trained encoder-decoder model. When used with a pre-trained T5, our
approach achieves new state-of-the-art results on WebNLG+2020 and
EventNarrative G2T generation datasets. Our method also shows to be very
effective in the low-resource setting.
