  Despite the success of diffusion models (DMs), we still lack a thorough
understanding of their latent space. While image editing with GANs builds upon
latent space, DMs rely on editing the conditions such as text prompts. We
present an unsupervised method to discover interpretable editing directions for
the latent variables $\mathbf{x}_t \in \mathcal{X}$ of DMs. Our method adopts
Riemannian geometry between $\mathcal{X}$ and the intermediate feature maps
$\mathcal{H}$ of the U-Nets to provide a deep understanding over the
geometrical structure of $\mathcal{X}$. The discovered semantic latent
directions mostly yield disentangled attribute changes, and they are globally
consistent across different samples. Furthermore, editing in earlier timesteps
edits coarse attributes, while ones in later timesteps focus on high-frequency
details. We define the curvedness of a line segment between samples to show
that $\mathcal{X}$ is a curved manifold. Experiments on different baselines and
datasets demonstrate the effectiveness of our method even on Stable Diffusion.
Our source code will be publicly available for the future researchers.
