  Graph neural networks (GNNs) have attracted much attention because of their
excellent performance on tasks such as node classification. However, there is
inadequate understanding on how and why GNNs work, especially for node
representation learning. This paper aims to provide a theoretical framework to
understand GNNs, specifically, spectral graph convolutional networks and graph
attention networks, from graph signal denoising perspectives. Our framework
shows that GNNs are implicitly solving graph signal denoising problems:
spectral graph convolutions work as denoising node features, while graph
attentions work as denoising edge weights. We also show that a linear
self-attention mechanism is able to compete with the state-of-the-art graph
attention methods. Our theoretical results further lead to two new models,
GSDN-F and GSDN-EF, which work effectively for graphs with noisy node features
and/or noisy edges. We validate our theoretical findings and also the
effectiveness of our new models by experiments on benchmark datasets. The
source code is available at \url{https://github.com/fuguoji/GSDN}.
