  Recent years have witnessed significant progress in developing effective
training and fast sampling techniques for diffusion models. A remarkable
advancement is the use of stochastic differential equations (SDEs) and their
marginal-preserving ordinary differential equations (ODEs) to describe data
perturbation and generative modeling in a unified framework. In this paper, we
carefully inspect the ODE-based sampling of a popular variance-exploding SDE
and reveal several intriguing structures of its sampling dynamics. We discover
that the data distribution and the noise distribution are smoothly connected
with a quasi-linear sampling trajectory and another implicit denoising
trajectory that even converges faster. Meanwhile, the denoising trajectory
governs the curvature of the corresponding sampling trajectory and its various
finite differences yield all second-order samplers used in practice.
Furthermore, we establish a theoretical relationship between the optimal
ODE-based sampling and the classic mean-shift (mode-seeking) algorithm, with
which we can characterize the asymptotic behavior of diffusion models and
identify the empirical score deviation.
