  Auto-encoders have emerged as a successful framework for unsupervised
learning. However, conventional auto-encoders are incapable of utilizing
explicit relations in structured data. To take advantage of relations in
graph-structured data, several graph auto-encoders have recently been proposed,
but they neglect to reconstruct either the graph structure or node attributes.
In this paper, we present the graph attention auto-encoder (GATE), a neural
network architecture for unsupervised representation learning on
graph-structured data. Our architecture is able to reconstruct graph-structured
inputs, including both node attributes and the graph structure, through stacked
encoder/decoder layers equipped with self-attention mechanisms. In the encoder,
by considering node attributes as initial node representations, each layer
generates new representations of nodes by attending over their neighbors'
representations. In the decoder, we attempt to reverse the encoding process to
reconstruct node attributes. Moreover, node representations are regularized to
reconstruct the graph structure. Our proposed architecture does not need to
know the graph structure upfront, and thus it can be applied to inductive
learning. Our experiments demonstrate competitive performance on several node
classification benchmark datasets for transductive and inductive tasks, even
exceeding the performance of supervised learning baselines in most cases.
