  In recent years, diffusion models have achieved remarkable success in various
domains of artificial intelligence, such as image synthesis, super-resolution,
and 3D molecule generation. However, the application of diffusion models in
graph learning has received relatively little attention. In this paper, we
address this gap by investigating the use of diffusion models for unsupervised
graph representation learning. We begin by identifying the anisotropic
structures of graphs and a crucial limitation of the vanilla forward diffusion
process in learning anisotropic structures. This process relies on continuously
adding an isotropic Gaussian noise to the data, which may convert the
anisotropic signals to noise too quickly. This rapid conversion hampers the
training of denoising neural networks and impedes the acquisition of
semantically meaningful representations in the reverse process. To address this
challenge, we propose a new class of models called {\it directional diffusion
models}. These models incorporate data-dependent, anisotropic, and directional
noises in the forward diffusion process. To assess the efficacy of our proposed
models, we conduct extensive experiments on 12 publicly available datasets,
focusing on two distinct graph representation learning tasks. The experimental
results demonstrate the superiority of our models over state-of-the-art
baselines, indicating their effectiveness in capturing meaningful graph
representations. Our studies not only provide valuable insights into the
forward process of diffusion models but also highlight the wide-ranging
potential of these models for various graph-related tasks.
